Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at

Bug 1239815

Summary: pypy: FTBFS in rawhide
Product: [Fedora] Fedora Reporter: Dennis Gilmore <dennis>
Component: pypyAssignee: Michal Cyprian <mcyprian>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 23CC: matti.picus, mcyprian, mstuchli, ndbecker2, tomspur, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: pypy-2.6.0-3.fc24 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-08-20 12:02:42 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1247795    
Bug Blocks: 1239338    
Description Flags
spec diff for PyPy 2.6.0 none

Description Dennis Gilmore 2015-07-05 21:13:51 UTC
Your package pypy failed to build from source in current rawhide.

For details on mass rebuild see

Comment 1 Dennis Gilmore 2015-07-05 21:13:52 UTC
Created attachment 1047760 [details]

Comment 2 Dennis Gilmore 2015-07-05 21:13:53 UTC
Created attachment 1047761 [details]

Comment 3 Dennis Gilmore 2015-07-05 21:13:54 UTC
Created attachment 1047762 [details]

Comment 4 Jan Kurik 2015-07-15 13:32:03 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 23 development cycle.
Changing version to '23'.

(As we did not run this process for some time, it could affect also pre-Fedora 23 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 23 End Of Life. Thank you.)

More information and reason for this action is here:

Comment 5 mattip 2015-08-04 14:58:16 UTC
We had a discussion with a centos person on IRC, and got a spec together that successfully builds pypy 2.6.0 on centos. I don't know enough to submit a patch that would pass tests, but the issues we encountered are:
- building PyPy 2.6.0 requires python 2.7 (or a pypy python)
- the set of import commands, which used to be required to enable building and installing cffi import libraries, is not part of the translation/complilation process
- PyPy builds the lions share of its code as, and then adds a thin executable to call into the shared object. This enables embedding PyPy in other programs. Ideally this shared object would be in some system wide library, but the build process patches a RPATH into the executable so that the so can be found if it is located in the same directory as the executable. This does not play well with soft-linking or hard-linking the executable into another directory.

FWIW, I am uploading the diff to the spec, it is very rough (disables tests) but should give someone an idea of what needs to be done

Comment 6 mattip 2015-08-04 15:00:27 UTC
Created attachment 1059119 [details]
spec diff for PyPy 2.6.0

A rough first cut actually contributed by a CentOS user, posted here with permission.

Comment 7 Zbigniew Jędrzejewski-Szmek 2015-08-04 15:15:13 UTC
python-devel currently requires python, so the no change to requires is necessary (for Fedora).

Are you disabling tests because they fail, or for some other reason?

Comment 8 mattip 2015-08-04 15:33:10 UTC
sorry, that is part of the diff that I'm not so sure about. It does seem wrong, tests should all pass but are quite time consuming (could require hours)

Comment 9 Matej Stuchlik 2015-08-05 19:48:22 UTC
Thanks Matti!
Michal Cyprian (CC'd) is currently working on fixing the FTBFS and using upstream, but it's taking a while, since Michal is new to the giant of pypy.spec. :)

(In reply to mattip from comment #5)
> - the set of import commands, which used to be required to enable building
> and installing cffi import libraries, is not part of the
> translation/complilation process

We've been having trouble with this particular part:

File "<stdin>", line 1, in <module>
File "/usr/lib64/pypy-2.6.0/lib_pypy/",
line 16, in <module>
from _syslog_cffi import ffi, lib
ImportError: No module named _syslog_cffi

Have you not run into similar issues?

Comment 10 Zbigniew Jędrzejewski-Szmek 2015-08-05 20:17:43 UTC
I think it might be fixed post 2.6.0 release. When building the rpm I have:

[translation:info] copied: /builddir/build/BUILD/pypy-2.6.0-src/
[translation:info] usession directory: /builddir/build/BUILD/pypy-2.6.0-src/usession-pypy-0
[translation:info] created: /builddir/build/BUILD/pypy-2.6.0-src/pypy-c
[77918] translation-task}
[Timer] Timings:
[Timer] annotate                       ---  551.6 s
[Timer] rtype_lltype                   --- 1033.9 s
[Timer] pyjitpl_lltype                 --- 1329.5 s
[Timer] backendopt_lltype              ---  340.6 s
[Timer] stackcheckinsertion_lltype     ---  317.7 s
[Timer] database_c                     ---  452.0 s
[Timer] source_c                       ---  861.0 s
[Timer] compile_c                      ---  751.7 s
[Timer] ===========================================
[Timer] Total:                         --- 5638.0 s
real    94m54.822s
user    80m57.411s
sys     1m20.253s
+ echo --------------------------------------------------------------

but when running the same command in hg tip I have:

[translation:info] copied: /home/zbyszek/pypy/
[translation:info] usession directory: /home/zbyszek/pypy/usession-default-0
[translation:info] created: /home/zbyszek/pypy/pypy-c
[54921] translation-task}
[translation:info] Create cffi bindings for modules...
[54921] {translation-task
starting build_cffi_imports
* _tkinter/
/home/zbyszek/pypy/pypy-c successfully built, but errors while building the above modules will be ignored
[54923] translation-task}
[Timer] Timings:
[Timer] annotate                       ---  558.0 s
[Timer] rtype_lltype                   ---  762.8 s
[Timer] pyjitpl_lltype                 ---  943.0 s
[Timer] backendopt_lltype              ---  256.2 s
[Timer] stackcheckinsertion_lltype     ---  161.9 s
[Timer] database_c                     ---  351.9 s
[Timer] source_c                       ---  408.9 s
[Timer] compile_c                      ---  665.6 s
[Timer] build_cffi_imports             ---    2.8 s
[Timer] ===========================================
[Timer] Total:                         --- 4111.0 s

Comment 11 Zbigniew Jędrzejewski-Szmek 2015-08-05 20:22:14 UTC
changeset:   78323:85bc12fb4725
branch:      py3k
parent:      78182:ae79f5787d65
parent:      78314:6d7ee832797e
user:        Manuel Jacob <me>
date:        Fri Jun 26 16:26:06 2015 +0200
summary:     hg merge default

+        # ugly hack to modify target goal from compile_c to build_cffi_imports
+        # this should probably get cleaned up and merged with driver.create_exe
+        from rpython.translator.driver import taskdef
+        import types
+        class Options(object):
+            pass
+        def mkexename(name):
+            if sys.platform == 'win32':
+                name ='exe')
+            return name
+        @taskdef(['compile_c'], "Create cffi bindings for modules")
+        def task_build_cffi_imports(self):
+            from pypy.tool.build_cffi_imports import create_cffi_import_libraries
+            ''' Use cffi to compile cffi interfaces to modules'''
+            exename = mkexename(driver.compute_exe_name())
+            basedir = exename
+            while not basedir.join('include').exists():
+                _basedir = basedir.dirpath()
+                if _basedir == basedir:
+                    raise ValueError('interpreter %s not inside pypy repo', 
+                                     str(exename))
+                basedir = _basedir
+            modules = self.config.objspace.usemodules.getpaths()
+            options = Options()
+            # XXX possibly adapt options using modules
+            failures = create_cffi_import_libraries(exename, options, basedir)
+            # if failures, they were already printed
+            print  >> sys.stderr, str(exename),'successfully built, but errors while building the above modules will be ignored'
+        driver.task_build_cffi_imports = types.MethodType(task_build_cffi_imports, driver)
+        driver.tasks['build_cffi_imports'] = driver.task_build_cffi_imports, ['compile_c']
+        driver.default_goal = 'build_cffi_imports'
+        # HACKHACKHACK end

Comment 12 mattip 2015-08-05 20:26:43 UTC
That is part of the cffi changes I alluded to above. Since cffi 1.0 (an integral part of pypy 2.6.0) there is a separation between runtime and buildtime. At buildtime, pypy builds import libraries for sqlite3, audioop, tk, curses, syslog, gdbm and pwdgrp via a script in pypy/tool/build_cffi_imports. For reference, in pre-cffi-1.0, the runtime libraries were built by importing, as root, the respective modules. That is why the new spec deleted those lines

In pypy 2.6.0 and onwards the script is run both by the translation process and by, as Zbigniew pointed out.

The script creates import libraries in <pypy_sources>/lib_ppy, on my system (ubuntu, sorry) I have these files after translation:

These must be installed as part of the packaging of pypy, probably to /usr/lib64/pypy-2.6.0/lib_pypy ?

Comment 13 mattip 2015-08-05 20:28:36 UTC
(In reply to mattip from comment #5)

> - the set of import commands, which used to be required to enable building
> and installing cffi import libraries, is not part of the
> translation/complilation process

GACK, that should be "is now part of the translation/complilation process" 

Kind of changes the meaning, no?

Comment 14 mattip 2015-08-05 20:34:26 UTC
I see that we actually made building the import libraries part of the translation process only after releasing 2.6.0, so for the 2.6.0 release you _must_ run in order to build the import libraries. Sorry for the mess

Comment 15 Michal Cyprian 2015-08-20 12:02:42 UTC
I've finally finished spec file using in %install section. It took me more time than I expected. If you have some suggestions or recommendations let me know.