Monday, June 11, 2012

How-to: using talend open studio for big data tutorial-2, read Hive data

Once the data was loaded to HDFS, let’s create hive external table by using the hive shell.
   fist make sure data is there,
   image

Then using the hive shell to create one external table named customers
   

CREATE EXTERNAL TABLE customers(id INT, name STRING,
level STRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\;'
STORED AS TEXTFILE
LOCATION '/usr/talend';

image
in order for the talend to connect to the hive, we need to start one hive thrift server.
image

 

Now let’s create one flow to read the data in hive. pick up the version ,and the thrift server ip/port, then put a hive query.
image

once import step, click the edit scheme and just add one column with type as object. then we will parse the result and map to our schema.
image

Click the advanced tab, to enable the parse query results, using the column we just created as object type.
image

drag the parserecordset component to the surface and conenct the mainout of hiverow to it, click edit schemas to do the mapping.
image

then match the values,
image

the the job looks like the following,
image

if we need to output the row, just put one logrow and connect, the final layout,
image

Now click to run this job, from the console it tell you whether it has connected to the hive server successfully, if failed, why.
image
on the hive server, it will show you it receive one query and parse it to mapreduce on the fly,
image
once done, you can see the results from the run console.
image

once done, we can export the result to Hbase to enable real-time application query. check it here,

21 comments:

mna said...

good tutorial!!!
i want to ask about "pick up the version ,and the thrift server ip/port", from where can i do it? i'am hadoop newbie. thank you

Unknown said...

QuickBooks software helps the businesses to run smoothly either small size business or mid size businesses. QuickBooks also helps you with your confusions by just contacting our QuickBooks Enterprise Support Number +1-833-400-1001.This software provides you the best possible assistance and keeps you updated about each and every details about your business. Further we are available 24/7 for your help for any kind of assistance you just need to contact our QuickBooks Enterprise Support +1-833-400-1001. QuickBooks regularly updates itself according to the requirements of the users and we are always ready to add more features to help you in more and more convenient ways so that you can run your business smoothly and grow your business . This is our basic and most important concern about the QuickBooks users , help the QuickBooks users and create a healthy environment for them to run and grow their business smoothly . In any case you need our help you are welcome to contact our QuickBooks team by calling us on QuickBooks Enterprise Support Phone Number +1-833-400-1001.We are at your service 24/7 feel free to contact us regarding any issue regarding QuickBooks.

Alina Harper said...

Cashless is the new yes, dial cash app customer service for more

Cashless is trending all across the globe. You too should join the chain with no fear. You don’t have to worry about the insecurities of using digital currency. You just have to dial us at our toll-free number and talk to the techie at cash app Customer service and get your problems solved. So, you can easily join the cashless army. Join now.
http://bit.ly/34mXEBq

Alina Harper said...

Transfer money to your bank account from app, call cash app customer service for help

You can easily transfer funds from your cash app to your bank account and get use your money anywhere. If you face any kind of problem while doing this, always remember, the techie sat cash app Customer service are available at your service. They will help you solve all kind of issues. you just have to let them know your problem at our toll-free number .
http://bit.ly/35G8z9g

Alina Harper said...

How To Have Better Knowhow? Dial Cash App Customer Service Number

Do you want to have a better knowledge of cash app? Don’t you know that how to make use of cash app? If yes, then you are not required to take tension. Just grab our
cash app Customer service by dialing our toll-free phone number which can be used by anyone at anytime from anywhere the globe.
http://bit.ly/2KZ1j0p

Digital Support said...

Gmail Customer Service: An Immediate Aid For Storage Problems
Make use of Gmail Customer Service right now and share the storage related problems with the professionals you have approached through the above mode. Whether it is technical inconvenience or any complex mishap causing storage related hurdles, you will surely resolve all your problems with optimum ease.

Digital Support said...

Doorstep Gmail Support Service Now Coming At Cost Effective Service Charges
You don’t need to have a financial burden while availing the best in class Gmail Support service as the charge of availing the same service is very low and can be availed at anytime from anywhere. Round the clock availability of such support service will help to have a hassle free experience on Gmail.

Digital Support said...

Take Gmail Support Service If Unable To Chat On Hangouts
You are not required to take stress or worry about the problems you face when it comes to chatting with any of your friends on Gmail hangouts. Whenever such problems take place, consider taking Gmail Support Service would be the right decision and also provides you with the dependable solution in no time.

Digital Support said...

Check Out The Gmail Customer Service Facility For Gmail Drive Problems
Are you facing problems with the Gmail drive? Are you not capable of accessing Gmail Drive in order to safekeeping of any essential data, emails and documents? The Gmail Customer Service is the round the clock working facility that one can avail for the purpose of fixing such problems from the root.

Digital Support said...

Grab Reasonable Technical backing at Gmail Phone Number
Are you stuck during the course of repairing the problematic Google mail account? In such a critical case, you should avoid the problems due to which you have been facing a lot of hurdles and obstacles. For the best resolution, you can try using Gmail Phone Number at anytime from anywhere in the world.

Digital Support said...

Mitigate Complex Security Shortcomings And Loopholes By Using Gmail Phone Number
Do you want to put all Gmail security hurdles aside? For that, you can avail technical support by just making use of Gmail Phone Number and the all your loopholes and shortcomings sorted out within a few seconds. Here, safety tips would also be given so that you could safeguard your Google mail account.

Digital Support said...

How Does Gmail Help To Create New Label For Emails?
Are you completely unable to create new label for your Google mail account? Are you worried just because of the same? In such a case, you should quickly approach the industrious professionals using Gmail Help and support service and opt for the reliable possible backing in a couple of seconds.

Josh Buttlar said...


Facebook Help: A Way To Get You Locked Account Unlocked

If your Facebook account has been locked due to various reasons and now you are unable to get the locked account unlocked, you should opt for Facebook Help. For that, you should immediately get in touch with the highly professional geeks who will surely direct you in every possible manner. Read More:- https://www.contact-customerservice.net/contact-facebook-help-center/

Josh Buttlar said...


Troubleshoot Complex Issues with Cash App Support Services

Cash App users often come across problems in their account and need someone to troubleshoot complex issues within a given dedicated time frame.
Cash App Support Services take great initiative in handling all technical matters on a serious note and shows no stone unturned in handling any technical issues as compared to our technical support.
Read More:- https://www.customerservicephone-number.com/cash-app-customer-service/

Josh Buttlar said...

Take Proper Assistance For Rectifying ATT Yahoo Mail Login Hurdles

You should make use of such mode of getting the problems solution which can be accessible at anytime from anywhere. If you come across Yahoo Mail Login problems and hurdles, you can get the right solution to it. For the optimum solutions on other Yahoo problems, it is surely the best option.
Read More:- https://www.emailcontacthelp.com/blog/att-yahoo-email-login/

Josh Buttlar said...


Facebook Help: A Way To Get You Locked Account Unlocked

If your Facebook account has been locked due to various reasons and now you are unable to get the locked account unlocked, you should opt for Facebook Help. For that, you should immediately get in touch with the highly professional geeks who will surely direct you in every possible manner. Read More:- https://www.contact-customerservice.net/contact-facebook-help-center/

Josh Buttlar said...

Troubleshoot Complex Issues with Cash App Support Services

Cash App users often come across problems in their account and need someone to troubleshoot complex issues within a given dedicated time frame.
Cash App Support Services take great initiative in handling all technical matters on a serious note and shows no stone unturned in handling any technical issues as compared to our technical support.
Read More:- https://www.customerservicephone-number.com/cash-app-customer-service/

Check Walmart Card said...

You can now check target gift card balance online by just entering the 16-digit gift card number and 3-digit CVV gift card access number and click to check target visa gift card balance.

check target gift card balance
target check balance
check target card balance
target gift card balance
check target balance
target visa gift card balance
check target visa gift card balance
target visa card balance
target gift card balance online
target gift card balance check
check target visa balance
check my target gift card balance
target gift card amount value
target gift balance
target gift card balance check online
target card balance
target check balance
target visa balance
target visa prepaid card balance
mybalancenow target
check mybalancenow
target prepaid visa balance
visa prepaid card balance target
target visa check balance

Rory Raine said...

Gift Cards And Business Gift Cards. check American express gift card balance. To view your available American express prepaid card balance and transaction history, please enter the information below and click Amex gift card balance check.

american express gift card balance
amexgiftcard com balance
check american express gift card balance
amex gift card balance check
express gift card balance
american express prepaid card balance
american express gift card activation
amex gift balance
american express card balance
american express check balance
american express prepaid gift card balance
amex check balance

Target Visa Gift Card Balance Check said...

Check your target visa gift card balance on my gift card online. Each time you use your visa prepaid card balance target, the amount of the purchase is deducted from the available balance on the card.

target visa gift card balance
target visa balance
target visa card balance
check target visa gift card balance
target visa prepaid card balance
mybalancenow target
check mybalancenow
target prepaid visa balance
visa prepaid card balance target
target visa check balance

Best Buy Gift Card Balance said...

How do I check best buy gift card balance? In Checkout, enter both the card number and 16-digit Number on the back of the card CVV number and click best buy gift card balance online.

best buy gift card balance online
best buy gift card balance
check best buy gift card balance
best buy card balance
best buy balance check
best buy card balance check
best buy gift card value
check my best buy gift card balance
best buy gift card lookup
best buy gift card amount
best buy gift balance
best buy gift card balance check online
best buy store credit balance
best buy gift card balance online

best buy gift card balance
check best buy gift card balance
best buy card balance
best buy balance check
best buy card balance check
best buy gift card value
check my best buy gift card balance
best buy gift card lookup
best buy gift card amount
best buy gift balance
best buy gift card balance check online
best buy store credit balance
best buy gift card balance online

best buy gift card balance
check best buy gift card balance
best buy card balance
best buy balance check
best buy card balance check
best buy gift card value
check my best buy gift card balance
best buy gift card lookup

 
Locations of visitors to this page