Streaming Twitter Data Analysis Using Spark for Effective Job Search

Near real time Big Data from social network sites like Twitter or Facebook has been an interesting source for analytics by researchers in recent years owing to various factors including its up-to-date-ness, availability and popularity, though there may be a compromise in genuineness or accuracy. Apache Spark, the trendy big data processing engine that offers faster solutions compared to Hadoop, can be effectively utilized in finding patterns of relevance useful for the common man from these sites. Recently many organizations are advertising their job vacancies through tweets, which saves time and cost in recruitment. This paper addresses the issue of real time analyzing and filtering those numerous job advertisements from among the millions of other streaming tweets and classify them into various job categories to facilitate effective job search, utilizing Spark.