Wednesday, November 16, 2011

How to: Install and Test apache mahout on hadoop

Mahout and Hadoop are all java libraries basically, mahout use the Maven tool to build the source code and maintain the dependency.
So we need make sure we have the following bits ready.

  • JDK
  • Maven
  • Hadoop
  • Mahout

 

I will start from the fresh centos, then get all those stuff ready step by step.

install JDK.
GO to Oracle JDK download site, http://www.oracle.com/technetwork/java/javase/downloads/index.html, I still prefer the Java 6 instead of 7, pickup one the .bin link, download and run it directly. I will put the java under /usr/lib/jdk6 folder.
image 
Export the bin directory to PATH, and jdk6 to JAVA_HOME environment variable.

Install MAVEN
Download the binary package from http://maven.apache.org/download.html, here I chose the 2.2.1 version which is more stable. 
image

Extract the zip file, and link it to /usr/lib/maven, then.
Export /usr/lib/maven to the PATH. now, you can run mvn –version, to make sure it works at least we can get the version,
image
For settings like proxy, check it out here, http://maven.apache.org/download.html#Maven_Documentation

Install HADOOP.
you can check this out, if you want to install hadoop as a fully distributd mode, How to: install and config hadoop to run in fully distributed Mode, Centos.

here we just have one vm, so keep it easy for the mahout testing. I will use the Cloudera distribution,

Download the repo file for centos 5, http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo and copy it to yum repo directory.
imageNow just search hadoop, you will see all the components,we will use the hadoop-0.20-conf-pseudo one.
yum install  hadoop-0.20-conf-pseudo

image

once done, go to /usr/lib/hadoop/conf directory, change the java home to /usr/lib/jdk6 in hadoop-env.sh

image

Then run as hdfs, format the namenode,

image
then start those daemons like /etc/init.d/hadoop-*, run JPS, you should see all the java process there,
image

now we can run a simple test, go the /usr/lib/hadoop, run

image
we can just copy one file there,
image
open a browser, go to http://localhost:50070, you can see the file we just uploaded is there,
image
Now , HDFS is ready. we can run a mapreduce job to make sure hadoop is ready.
image

If no error, we are all set, hadoop is ready.

Install Mahout.

Download one source code, you can use svn to clone one trunk copy,

svn co http://svn.apache.org/repos/asf/mahout/trunk



























and copy this code to /usr/lib/mahout

then run mvn install –DskipTests to compile the source, mvn will figure out the dependency and fetch those jars for you automatically,


image





it takes time to download all the jar, mileage depends.


Here is my MPG, take several minutes,


image





Now, export /usr/lib/mahout/bin to PATH , then we can run mahout from the shell.


If you cant exectute the mahout, give it one execute permission.


run mahout, will list all the options to go with different algorithms.


image



Then go the examples folder, run mvn compile



image



Now, you can run some example like the one to classify the news groups.



image





Here we didn’t sepecify the HADOOP_HOME, so it will run locally. the shell script will download data, prepare the data, then run the classifier.



image



when done, it will show the confusion matrix against the testing data.

image

250 comments:

«Oldest   ‹Older   201 – 250 of 250
rocky said...

You are including better information regarding this topic in an effective way. thank you so much information.
Python Training in Chennai

Python Training in Bangalore

Python Training in Hyderabad

Python Training in Coimbatore

Python Training

python online training

python flask training

python flask online training


Sonu Kabir said...

Thanks for sharing information awesome blog post Online Education Quiz website For Exam Follow this website Gk in Hindi

Devi said...

Thanks for sharing the very useful info,Really Very Informative Blog. oracle training in chennai

deiva said...

Good written and great info. Thank you for taking the time to provide us with your valuable information. Please Keep it up...
web designing training in chennai

web designing training in omr

digital marketing training in chennai

digital marketing training in omr

rpa training in chennai

rpa training in omr

tally training in chennai

tally training in omr

Jayalakshmi said...

This is an awesome post. Really very informative and creative contents. These concept is a good way to enhance the knowledge.
sap training in chennai

sap training in tambaram

azure training in chennai

azure training in tambaram

cyber security course in chennai

cyber security course in tambaram

ethical hacking course in chennai

ethical hacking course in tambaram

praveen said...

Wonderful blog with great piece of information. Regards to your effort. Keep sharing more such blogs.Looking forward to learn more from you.
data science training in chennai

data science training in porur

android training in chennai

android training in porur

devops training in chennai

devops training in porur

artificial intelligence training in chennai

artificial intelligence training in porur

Arthur Rodriguez said...

HP printer Customer service number
HP Printer helpline phone number
HP printer drivers for windows 10
Printer drivers for HP printer
HP wireless printer drivers for Windows
www.aol.com/official site

Sam Miller said...

solutions.brother.com/windows
solutions.brother.com/windows
canon.com/ijsetup

shiny said...

I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's.


sap training in chennai

sap training in annanagar

azure training in chennai

azure training in annanagar

cyber security course in chennai

cyber security course in annanagar

ethical hacking course in chennai

ethical hacking course in annanagar

shiny said...

I am satisfied that you simply shared this helpful information with us. Please stay us informed like this. Thanks for sharing.

sap training in chennai

sap training in annanagar

azure training in chennai

azure training in annanagar

cyber security course in chennai

cyber security course in annanagar

ethical hacking course in chennai

ethical hacking course in annanagar

jeni said...

Outstanding blog post, I have marked your site so ideally I’ll see much more on this subject in the foreseeable future.
oracle training in chennai

oracle training in velachery

oracle dba training in chennai

oracle dba training in velachery

ccna training in chennai

ccna training in velachery

seo training in chennai

seo training in velachery


CloudLearn ERP said...

 You are really performing grand work. I must articulate that you really have done a great research before writing. Keep up the good work!
Data Science training in Mumbai
Data Science course in Mumbai
SAP training in Mumbai

Ramesh ji said...

We have seen many blog but this the great one, Thanks for provide great informatic and looking beautiful blog, really nice required information & the things i never imagined and i would like to request, wright more blog and blog post like that for us. Thanks you once agian

birth certificate in delhi
birth certificate in noida
birth certificate in ghaziabad
birth certificate in gurgaon
correction in birth certificate
marriage registration in delhi
marriage certificate delhi
how to change name in 10th marksheet
marriage registration in ghaziabad
marriage registration in gurgaon

ganesh said...

Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic.
Angular js Training in Chennai

Angular js Training in Velachery

Angular js Training in Tambaram

Angular js Training in Porur

Angular js Training in Omr
Angular js Training in Annanagar

harini said...

This blog is very interesting. I learned so much and want to thank you for sharing it in the first place. It is really helpful for my future endeavors. Thanks for your efforts and making it available to public
Selenium Training in Chennai

Selenium Training in Velachery

Selenium Training in Tambaram

Selenium Training in Porur

Selenium Training in Omr

Selenium Training in Annanagar

Arthur Rodriguez said...

www.canon.com/ijsetup
www.norton.com/setup
solutions.brother.com/windows
Malwarebytes unable to connect to service
virus alert from Microsoft this computer is blocked
avast virus definitions won't update

Arthur Rodriguez said...

canon.com/ijsetup
epson connect printer setup utility
www.avg.com/retail
how to install Norton antivirus on Windows 10
how do i reinstall norton 360 without disc
support.brother.com/windows
Canon printer drivers for chromebook

sahasrit said...

thanks for sharing information awesome blog post .
amazon web services aws training in chennai

microsoft azure training in chennai

workday training in chennai

android-training-in chennai

ios training in chennai

Anonymous said...

Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.
tally training in chennai

hadoop training in chennai

sap training in chennai

oracle training in chennai

angular js training in chennai

Hemachandran said...

Really it was an awesome article,very interesting to read.You have provided an nice article,Thanks for sharing.
Java course in chennai

python course in chennai

web designing and development course in chennai

selenium course in chennai

digital-marketing seo course in chennai

Anonymous said...

Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.
salesforce training in chennai

software testing training course in chennai

robotic process automation rpa training in chennai

blockchain training in chennai

devops training in chennai

Pushba said...

Your writing is very unique. Amazing post. It is very informative. Thanks for sharing.
IELTS Coaching in chennai

German Classes in Chennai

GRE Coaching Classes in Chennai

TOEFL Coaching in Chennai

Spoken english classes in chennai | Communication training

Mahawar Alloys said...

Web development course in Patiala
Ppc course in Patiala
Computer centre in patiala
Digital Marketing Course in Patiala
Web designing course in Patiala
Graphic designing course in Patiala
Seo course in Patiala
App development course in Patiala
Social media marketing course in Patiala

Ranjith said...

I think this the best blog with the most important information filled in this website
I Got the more and more ideas from this blog Thank You for letting us to know.
pythonTraining in chennai

python Course in chennai

jegan said...

wonderful article contains lot of valuable information. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article.
This article resolved my all queries.good luck an best wishes to the team members.learn digital marketing use these following link
Digital Marketing Course in Chennai

Prwatech said...

Wow, amazing post! Really engaging, thank you.

Python Training Institute in Pune
Best Python Classes in Pune

Tutorials said...

Very Informative and interesting blog!  Primavera course online  Best primavera p6 training online  

data scientist london said...

Wonderful blog found to be very impressive to come across such an awesome blog. I should really appreciate the blogger for the efforts they have put in to develop such an amazing content for all the curious readers who are very keen of being updated across every corner. Ultimately, this is an awesome experience for the readers. Anyways, thanks a lot and keep sharing the content in future too.
Digital Marketing Training

Huongkv said...

Mua vé máy bay tại Aivivu, tham khảo

Lịch bay từ Hàn Quốc về Việt Nam hôm nay

vé máy bay hà nội sài gòn vietjet

vé máy bay cần thơ hà nội bamboo

vé máy bay hải phòng nha trang

vé máy bay đi Huế Vietjet

taxi sân bay nội bài

Mallela said...

Thanks for posting the best information and the blog is very helpful.digital marketing institute in hyderabad

Jacob Grayson said...

www.canon.com/ijsetup
www.canon.com/ijsetup
www.canon.com/ijsetup
www.canon.com/ijsetup
www.canon.com/ijsetup
www.canon.com/ijsetup
www.canon.com/ijsetup
www.canon.com/ijsetup

Jacob Grayson said...

solutions.brother.com/windows
solutions.brother.com/windows
solutions.brother.com/windows
solutions.brother.com/windows
solutions.brother.com/windows

ram said...

Sharing the same interest, Infycle feels so happy to share our detailed information about all these courses with you all! Do check them out
Best Data training in chennai & get to know everything you want to about software trainings.

Devi said...

Reach to the best software training institute in Chennai, Infycle Technologies, to enter the IT industry with well-defined skills. Infycle Technologies is the rapidly developing software training cum placement center in Chennai and is generally known for its significance in providing quality hands-on practical training with 200% guaranteed outcomes! Call 7502633633 to book a free demo and to avail the best offers.Best Software Training Institute in Chennai | Infycle Technologies

Target Visa Gift Card Balance Check said...

Thanks for sharing. I found a lot of interesting information here. A really good post, very thankful and hopeful that you will write many more posts like this one.

check target visa gift card balance,
target visa gift card balance,

Mallela said...

Thanks for posting the best information and the blog is very important.digital marketing institute in hyderabad

Locas watson said...

Thanks for sharing the details. I simply wish to let you understand that I really looked into your site and also I found it fascinating and useful.
Sephora Gift Card Balance Sephora Gift Card Balance Check

nayar said...

Wonderful blog.Thanks for sharing such a worthy information...
Digital Marketing Courses in Bangalore
Digital Marketing Course in Pune
Digital Marketing Course in Hyderabad
Digital Marketing Course in Gurgaon

Technogeekscs said...

Fantastic blog! Thanks for sharing a very interesting post, I appreciate to blogger for an amazing post.

Devops Training Institute in Pune
Devops Training in Pune

KITS ONLINE TRAINING said...

abinitio training
tableau training
servicenow training
vmware training
tableau training
sccm training

Free CSEET Online Classes said...

Are you getting bored during this lockdown ? We are here for you to kill your boredom and offering amazing courses for you only. There will be CS executive classes which is very easy to understand and a free of cost CSEET classes so guys hurry up . To avail this offer visit our website or contact us at https://uniqueacademyforcommerce.com/

Toffee said...

Great post. keep sharing such a worthy information

Ethical Hacking Course in Chennai
Ethical Hacking course in Bangalore

ram said...

Finish the Get Big Data Certification in Chennai from Infycle Technologies, the best software training institute in Chennai which is providing professional software courses such as Data Science, Artificial Intelligence, Java, Hadoop, Selenium, Android, and iOS Development, etc with 100% hands-on practical training. Dial 7502633633 to get more info and a free demo and to grab the certification for having a peak rise in your career.

ram said...

Infycle Technologies, the No.1 software training institute in Chennai offers the No.1 Big Data Hadoop Training in Chennai | Infycle Technologies for students, freshers, and tech professionals. Infycle also offers other professional courses such as DevOps, Artificial Intelligence, Cyber Security, Python, Oracle, Java, Power BI, Selenium Testing, Digital Marketing, Data Science, etc., which will be trained with 200% practical classes. After the completion of training, the trainees will be sent for placement interviews in the top MNC's. Call 7502633633 to get more info and a free demo.

kishor said...

nices information thanku so much
thanku so
click here

Data Science Training in Bhilai - 360DigiTMG said...

Impressive blog to be honest definitely this post will inspire many more upcoming aspirants. Eventually, this makes the participants to experience and innovate themselves through knowledge wise by visiting this kind of a blog. Once again excellent job keep inspiring with your cool stuff.

Data Science Training in Bhilai

Data Science Training in Bhilai - 360DigiTMG said...

Wonderful blog found to be very impressive to come across such an awesome blog. I should really appreciate the blogger for the efforts they have put in to develop such an amazing content for all the curious readers who are very keen of being updated across every corner. Ultimately, this is an awesome experience for the readers. Anyways, thanks a lot and keep sharing the content in future too.

Data Science Course in Bhilai

Online Helmet & Other Acessories Store said...

Shop for the latest Redmi mobiles from Helmet Don at the best prices in India. Xiaomi smartphones include Mi Series, Mi Note Series, Redmi Series, Pocophone, Mi Max Series, Mi Mix Series, and the Blackshark.
HelmetDon
dslr-camera
oppo-phones

Devi said...

Get the Digital Marketing Training in Chennai from Infycle Technologies, one of the best software training institute, and Placement center in Chennai which is providing professional software courses such as Data Science, Artificial Intelligence, Cyber Security, Big Data, Java, Hadoop, Selenium, Android, and iOS Development, DevOps, Oracle, etc with 200% hands-on practical training. Dial 7504633633 to get more info and a free demo and to grab the certification for having a peak rise in your career.Get Digital Marketing Course in Chennai | Infycle Technologies

Juliana petar said...



Trade Capital Options: TriumphFX Offers Customers The Opportunity To Trade A Wide Array Of Assets And Financial Instruments With A 30p Per Lot Commission.



















«Oldest ‹Older   201 – 250 of 250   Newer› Newest»
 
Locations of visitors to this page