In this post I will discuss four topics: 1) my journey to start with SparkNLP, 2) what went well, 3) what did not go well and, 4) getting certified.
My Journey to Start with SparkNLP
As a data science intern, I was tasked to complete a natural language processing project solely using SparkNLP. Therefore, I didn’t really have a choice to choose other approaches; however, after become familiar with the set up, I am really glad I used SparkNLP. There is no comparison regarding the amount of data that can be quickly processed using SparkNLP versus other approaches. It is truly remarkable how within a couple of minutes (or less), I can tokenize, lemmatize, remove stop words, and use a deep neural network to do named entity recognition.
What Went Well
John Snow Labs’ (JSL) SparkNLP Slack Channel saved my life. Maziyar, specifically, answered all my questions in a timely manner. I was amazed by his quick responses and willingness to help. Major props to JSL for setting that up.
What Did Not Go Well
In the workshop, I wish JSL used examples that had a dataframe of text instead of using one or two sentences as an example. Although, applying what they provided to a dataframe was not that difficult. In fact, I see that they just came out with a playground for dataframes.
To get certified I highly recommend to do the following: play around with SparkNLP for at least a week, attend their two-day workshop, study (rewatch the recorded lessons, go through their documentation), and take the exam as soon as possible while the material is still fresh in your mind. If you take my advice, I have no doubt you would pass the certification test.
I recommend to try using SparkNLP yourself before attending the workshop because Veysel, the instructor, goes through the material pretty quickly. I don’t blame him though because there is a lot to go over. Thankfully, Veysel is open to answering any questions both inside and outside of the workshop. He provided thorough explanations that quelled and confusion.
Overall, my experience with SparkNLP has been quite positive. I really appreciate their team’s willingness to help and am amazed by what can be done using SparkNLP!
YouTube link to a webinar JSL does hands-on coding to train a NER model from scratch
Workshop repo where you can start playing with Spark NLP in Colab
World Image : Creator: Qvasimodo | Credit: Getty Images/iStockphoto