Forwarding my reply with the solution to the problem to the list
The key was that I was testing with Pig's local execution mode so needed to use the
PIG_CLASSPATH environment variable instead
Rob
From: Rob Vesse <rvesse@yarcdata.com<mailto:rvesse@yarcdata.com>>
Date: Monday, 20 January 2014 16:04
To: "Datta, Kushal"
<kushal.datta@intel.com<mailto:kushal.datta@intel.com>>
Subject: Re: [GraphBuilder] ClassNotFound error trying to run rdf_example.pig
As I already said in my email I had done this. I've tried putting the GraphBuilder
JAR in both the HADOOP_CLASSPATH and the CLASSPATH and neither one makes any difference.
The key thing here appears to be that I am using Pig's local execution mode which
means neither of the above are honoured, changing to use PIG_CLASSPATH resolves this
problem
Rob
From: <Datta>, Kushal
<kushal.datta@intel.com<mailto:kushal.datta@intel.com>>
Date: Monday, 20 January 2014 13:53
To: GraphBuilder Users List
<graphbuilder@lists.01.org<mailto:graphbuilder@lists.01.org>>
Subject: Re: [GraphBuilder] ClassNotFound error trying to run rdf_example.pig
Hi Rob,
Please extend HADOOP_CLASSPATH with the path to the graphbuilder jar before you use the
library.
$ export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/path_to_your_graphbuilder_jar
Thanks,
-Kushal.
From: GraphBuilder [mailto:graphbuilder-bounces@lists.01.org] On Behalf Of Rob Vesse
Sent: Monday, January 20, 2014 1:52 PM
To: GraphBuilder Users List
Subject: [GraphBuilder] ClassNotFound error trying to run rdf_example.pig
I'm trying to run the examples with the 2.0 alpha build and I'm getting the
following error:
2014-01-20 13:42:13,101 [Thread-6] WARN org.apache.hadoop.mapred.LocalJobRunner -
job_local_0001
java.io.IOException: Could not find class
com.intel.hadoop.graphbuilder.graphelements.SerializedGraphElementStringTypeVids, while
attempting to de-serialize it
at org.apache.pig.data.BinInterSedes.readWritable(BinInterSedes.java:293)
at org.apache.pig.data.BinInterSedes.readDatum(BinInterSedes.java:422)
at org.apache.pig.data.BinInterSedes.readDatum(BinInterSedes.java:318)
at org.apache.pig.data.utils.SedesHelper.readGenericTuple(SedesHelper.java:144)
at org.apache.pig.data.BinInterSedes.readTuple(BinInterSedes.java:149)
at org.apache.pig.data.BinInterSedes.readTuple(BinInterSedes.java:1374)
at org.apache.pig.impl.io.NullableTuple.readFields(NullableTuple.java:55)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
at org.apache.hadoop.mapreduce.ReduceContext.nextKeyValue(ReduceContext.java:116)
at org.apache.hadoop.mapreduce.ReduceContext.nextKey(ReduceContext.java:92)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:175)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260)
Caused by: java.lang.ClassNotFoundException:
com.intel.hadoop.graphbuilder.graphelements.SerializedGraphElementStringTypeVids
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at org.apache.pig.data.BinInterSedes.readWritable(BinInterSedes.java:291)
This is when running in local mode using the following command:
pig –x local –f examples/rdf_example.pig
Note that I have double checked that the graph builder 2.0 with dependencies JAR is in the
class path (I've tried both with CLASSPATH and HADOOP_CLASSPATH)
Anyone else run into this error or have any thoughts on the cause?
Rob
Rob Vesse
Software Engineer
Mobile +44 (0) 7791 638 873 | rvesse@yarcdata.com<mailto:rvesse@yarcdata.com> |
Skype rvesse
One Kingdom Street | London | W2 6BD |
www.yarcdata.com<http://www.yarcdata.com/>
[Description: cid:image002.jpg@01CE3B8C.715B1D60]<http://www.yarcdata.com/>