Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1149295

Summary:	OOM during build on s390x
Product:	[Fedora] Fedora	Reporter:	Dan Horák <dan>
Component:	hadoop	Assignee:	Robert Rati <rrati>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	21	CC:	besser82, matt, rrati
Target Milestone:	---
Target Release:	---
Hardware:	s390x
OS:	Unspecified
Whiteboard:
Fixed In Version:	hadoop-2.4.1-5.fc21	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-11-01 17:00:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	467765

Description Dan Horák 2014-10-03 16:47:00 UTC

JVM gets into OOM during hadoop build on s390x

...
INFO] --- build-helper-maven-plugin:1.9:add-source (add-jsp-generated-sources-directory) @ hadoop-hdfs ---
[INFO] Source directory: /builddir/build/BUILD/hadoop-common-9e2ef43a240fb0f603d8c384e501daec11524510/hadoop-hdfs-project/hadoop-hdfs/target/generated-sources/java added.
[INFO] 
[INFO] --- hadoop-maven-plugins:2.4.1:protoc (compile-protoc) @ hadoop-hdfs ---
[INFO] 
[INFO] --- hadoop-maven-plugins:2.4.1:protoc (compile-protoc-datanode) @ hadoop-hdfs ---
[INFO] 
[INFO] --- hadoop-maven-plugins:2.4.1:protoc (compile-protoc-namenode) @ hadoop-hdfs ---
[INFO] 
[INFO] --- hadoop-maven-plugins:2.4.1:protoc (compile-protoc-qjournal) @ hadoop-hdfs ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hadoop-hdfs ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hadoop-hdfs ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 588 source files to /builddir/build/BUILD/hadoop-common-9e2ef43a240fb0f603d8c384e501daec11524510/hadoop-hdfs-project/hadoop-hdfs/target/classes
The system is out of resources.
Consult the following stack trace for details.
java.lang.OutOfMemoryError: Java heap space
	at java.util.HashMap.newNode(HashMap.java:1734)
	at java.util.HashMap.putVal(HashMap.java:630)
	at java.util.HashMap.put(HashMap.java:611)
...

for full log please see http://s390.koji.fedoraproject.org/koji/taskinfo?taskID=1575786


Version-Release number of selected component (if applicable):
hadoop-2.4.1-2.fc21



And with

diff --git a/hadoop.spec b/hadoop.spec
index f8c4301..c5cada6 100644
--- a/hadoop.spec
+++ b/hadoop.spec
@@ -602,6 +602,9 @@ opts="-j"
 opts="-j"
 %endif
 %endif
+%ifarch s390x
+export MAVEN_OPTS="-Xms2048M -Xmx4096M"
+%endif
 %mvn_build $opts -- -Drequire.snappy=true -Dcontainer-executor.conf.dir=%{_sysconfdir}/%{name} -Pdist,native -DskipTests -DskipTest -DskipIT
 
 # This takes a long time to run, so comment out for now

hadoop builds fine on both s390 and s390x - http://s390.koji.fedoraproject.org/koji/taskinfo?taskID=1576118

Comment 1 Dan Horák 2014-10-06 14:06:24 UTC

same problem is on ppc64le

Comment 2 Dan Horák 2014-10-10 08:14:30 UTC

ppc64le additionally needs

diff -up hadoop-common-project/hadoop-common/src/JNIFlags.cmake.ppc hadoop-common-project/hadoop-common/src/JNIFlags.cmake
--- hadoop-common-project/hadoop-common/src/JNIFlags.cmake.ppc	2014-10-10 07:17:31.770038082 +0000
+++ hadoop-common-project/hadoop-common/src/JNIFlags.cmake	2014-10-10 07:18:10.050038084 +0000
@@ -78,6 +78,8 @@ IF("${CMAKE_SYSTEM}" MATCHES "Linux")
         SET(_java_libarch "amd64")
     ELSEIF (CMAKE_SYSTEM_PROCESSOR MATCHES "^arm")
         SET(_java_libarch "arm")
+    elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(powerpc|ppc)64le")
+        set(_java_libarch "ppc64")
     ELSE()
         SET(_java_libarch ${CMAKE_SYSTEM_PROCESSOR})
     ENDIF()

because OpenJDK upstream decided to install JRE into ppc64 directory. Ideally Hadoop should use Java detection from cmake (FindJNI.cmake) directly.

successful scratch builds on ppc64/ppc64le =
http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=2141939

Comment 3 Fedora Update System 2014-10-10 11:47:33 UTC

hadoop-2.4.1-5.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/hadoop-2.4.1-5.fc21

Comment 4 Fedora Update System 2014-10-11 06:58:02 UTC

Package hadoop-2.4.1-5.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing hadoop-2.4.1-5.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-12632/hadoop-2.4.1-5.fc21
then log in and leave karma (feedback).

Comment 5 Fedora Update System 2014-11-01 17:00:40 UTC

hadoop-2.4.1-5.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.