Tuesday, May 17, 2016

Spark window function, failure: ``union'' expected but `(' found

this a very weird error when I try to run a simple window ranking, all looks good from the syntax perspective.
team=[("Lakers","WEST",29 ),("Golden State","WEST",89 ),
      ("MIA HEAT","EAST",79 ),("SAS","WEST",9 ),
      ("RAPTORS","EAST",29 ) ]

sql.createDataFrame(
    sc.parallelize(
        team).map(lambda x: Row(Team=x[0],Division=x[1], Score=x[2])))\
    .registerAsTable("team")

print sql.sql("SELECT team, division, score, rank() OVER (PARTITION BY division ORDER BY score desc)"              "  as rank FROM team").take(10)

And I got this errors complaining the syntax
4j.protocol.Py4JJavaError: An error occurred while calling o36.sql.
: java.lang.RuntimeException: [1.43] failure: ``union'' expected but `(' found

SELECT team, division, score, rank() OVER (PARTITION BY division ORDER BY score desc)  as rank FROM team
                                          ^
 at scala.sys.package$.error(package.scala:27)
 at org.apache.spark.sql.catalyst.AbstractSparkSQLParser.parse(AbstractSparkSQLParser.scala:36)
 at org.apache.spark.sql.catalyst.DefaultParserDialect.parse(ParserDialect.scala:67)
 at org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:211)
 at org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:211)

To Fix this, please make sure you are using HiveContext instead of SqlContext

15 comments:

Elizabeth said...

A control ping-pong table is nine legs in length by 5 legs wide.watch Roland Garros live

Elizabeth said...

It rests 30 creeps wide off the ground and is composed from Masonite and secured with an extraordinary region that prevents the ball from diminishing down because of rubbing.Roland Garros Direct

Elizabeth said...

The covering is generally green or turquoise in shading, however other proper shades are accessible, for example, dark. plunging the work area in 50 percent is a white net that seems like a golf net. french open live

Unknown said...

Thanks. This post saved me hours of over thinking

Rachel Bilson said...

Crackle, the most entertaining channel is now on Roku to entertain you. If you are new to this channel, use the page crackle.com/activate. Once if you activate the channel, the most interesting program collections are on its way. The categories include full-length movies, TV shows, and documentaries and much more.

marvinericksen said...

Hey, This is me Marvin Ericksen writing from Florida. So how’s life going-on, Feeling Enthusiastic or feeling dull. I felt boring with my life. So here I am and going to write my blogs. Actually I am really interested towards Reading and writing. But my all time hobbies are movies and TV Shows and Series.
My works:
psvue.com/activateroku

Rachel Bilson said...

It’s always my passion to write creative articles. Have worked on a lot of innovative and interesting topics. Read it and it’s informative. Recommend you to provide your feedback & suggestions to work on more titles. Contact me or reach me if you would like to know more about my profile.
My work:Pureflix on Roku

marvinericksen said...

Hey, This is me Marvin Ericksen writing from Florida. So how’s life going-on, Feeling Enthusiastic or feeling dull. I felt bored with my life. So here I am and going to write my blogs. Actually I am really interested in reading and writing. But my all-time hobbies are movies and TV Shows and Series.
My blog: cbs.com/roku

Alex williams said...

We support all types of HP printer troubleshooting and service. Just enter the model number of your printer in 123.hp.com/setup to identify the software and drivers your printer requires. Download and install it in your mac and 'Run' the file. The process is easy however if you have any doubts or queries regarding HP printers contact us.

greyhankin said...

It’s not a tough task to activate Roku. Understand the Roku activation steps. Connect the necessary hardware. It's the power and HDMI, USB cable that is required. Sign in to Roku account and move to the page Roku.com/link. To begin streaming, visit the channel store and start the channel search. Select and then add the channel to proceed. Want to know How to Activate Roku? Use the customer support number available on our webpage.

Garmin GPS said...

I know the value of Garmin Express and keep reading good content about the same. I found this blog very informative and it increased my understanding about garmin update to a great extent.

Garmin Nuvi said...

I did not know the entire benefits of Garmin Nuvi Gps. But since I have read this blog on map update, I have installed all available updates for my garmin device. I must say that this piece of blog has helped me improve navigation experience. I am not suggesting all my friends to read this blog and get Garmin map update with this team.

Antivirus said...

We did not know the all benefits of Panda Antivirus Support Number. I have read this blog on Antivirus I have installed all available updates. I must say that this piece of blog has helped me improving navigation experience. I am not suggesting all my friends to read this blog and get Antivirus Panda Support with this team. You can use Panda Antivirus to get panda help desk update. Be it Norton Internet security, panda, Kaspersky, McAfee, or any other antivirus.

Aiden Jangra said...

Great post! I am actually getting ready to across this information, is very helpful my friend. HP customer service phone number Thanks for sharing such an amazing blog. I really appreciated.

Roku Com Link said...

I appreciate your hard work. Keep posting new updates with us. This is really a wonderful post. Nice Blog Very interesting and useful information on your blog. Thanks for sharing the blog and this great information which is definitely going to help us. Know about Roku enter code.

 
Locations of visitors to this page