Hi Jamal..
So I am also new to graphbuilder.. So please take my advice from grain of
salt and wait for others to reply.
I just tried to run graphbuilder demoapps on pseduo-distributed mode and
here are my notes.
Also, I am on mac...
1) Set up pseudo-distributed node and lets say the hadoop is on
/Users/hduser/hadoop-x.y.z/
2) Next copy the graphbuilder in a folder where it can be accessed by
hduser.. I copied it on /tmp/graphbuilder
3) cd /tmp/graphbuilder .. --> %mvn package
4) From the web (wget or curl)
http://dumps.wikimedia.org/enwiki/latest/
enwiki-latest-abstract1.xml<http://dumps.wikimedia.org/enwiki/latest/e...
and save it on /tmp/wiki/
The whole dump is like 8gb.. so I used just one file instead
5) Now... go to your hduser account ( su - hduser and login to your hadoop
account)
6) cd /Users/hduser .. copy the wiki file to hdfs by /bin/hadoop dfs
-copyFrom Local /tmp/wiki /Users/hduser/wiki-input
Cross check bin/hadoop dfs -ls /Users/hduser
you should see wiki-input listed there
7) Now run it by...
bin/hadoop jar
/tmp/graphbuilder/target/graphbuilder-0.0.1-SNAPSHOT-hadoop-job.jar
com.intel.hadoop.graphbuilder.demoapps.wikipedia.linkgraph.LinkGraphEnd2End
2 /Users/hduser/wiki-input /Users/hduser/wiki-output
This should run the app .
The only issue I am facing is where is the output... I see in one of the
steps.. hadoop throws an error that it failed to create an intermediatory
output
NFO edge.EdgeIngressMR: gzip = false
12/12/24 21:19:22 INFO edge.EdgeIngressMR:
===============================================================
12/12/24 21:19:22 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
12/12/24 21:19:22 INFO mapred.JobClient: Cleaning up the staging area
hdfs://localhost:9000/tmp/hadoop-hadoop/mapred/staging/hadoop/.staging/job_201212242100_0003
*12/12/24 21:19:22 ERROR security.UserGroupInformation:
PriviledgedActionException as:hadoop
cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not
exist: hdfs://localhost:9000/Users/hadoop/wiki-output/graph_norm/vdata*
*Input path does not exist:
hdfs://localhost:9000/Users/hadoop/wiki-output/graph_norm/edata*
I am not sure what was the issue..
If you or someone can help me with that.. it would be great.
Hope this helps.
On Mon, Dec 24, 2012 at 12:30 AM, jamal sasha <jamalshasha(a)gmail.com> wrote:
hi,
I am actually new to java.. so maybe thats the issue.
but basically i downloaded graphbuilder code.. and then did
mvn package
and then downloaded a wiki dataset
I am getting the error
*%~/Desktop/hadoop-1.0.1/bin/hadoop
target/graphbuilder-0.0.1-SNAPSHOT-hadoop-job.jar
com.intel.hadoop.graphbuilder.demoapps.wikipedia.linkgraph.LinkGraphEnd2End
1 ~/Desktop/graphbuilder-beta/wiki_data/ ~/Desktop/graphbuilder-beta/output
*
Exception in thread "main" java.lang.NoClassDefFoundError:
target/graphbuilder-0/0/1-SNAPSHOT-hadoop-job/jar
Caused by: java.lang.ClassNotFoundException:
target.graphbuilder-0.0.1-SNAPSHOT-hadoop-job.jar
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
_______________________________________________
GraphBuilder mailing list
GraphBuilder(a)lists.01.org
https://lists.01.org/mailman/listinfo/graphbuilder
--
Mohit
"When you want success as badly as you want the air, then you will get it.
There is no other secret of success."
-Socrates