Courses
Books
Blog
My Courses / Log In
Contact
Help
Cart

[Exercise] Implement Friends by Age with Datasets

Back to: Apache Spark 3 with Scala – Hands On with Big Data!

Previous Lesson

[Activity] Using DataFrames and DataSets

Exercise Solution

Leave a Reply Cancel reply

You must be logged in to post a comment.

Lessons

Getting Started
- Introduction, and Getting Set Up
- Introducing Apache Spark
Scala Crash Course [Optional]
- [Activity] Scala Basics
- [Exercise] Flow Control in Scala
- [Exercise] Functions in Scala
- [Exercise] Data Structures in Scala
Spark Basics and the RDD Interface
- The Resilient Distributed Dataset
- Ratings Histogram Walkthrough
- Spark Internals
- Key / Value RDD’s, and the Average Friends by Age example
- [Activity] Running the Average Friends by Age Example
- Filtering RDD’s, and the Minimum Temperature by Location Example
- [Activity] Running the Minimum Temperature Example, and Modifying it for Maximum
- [Activity] Counting Word Occurrences using Flatmap()
- [Activity] Improving the Word Count Script with Regular Expressions
- [Activity] Sorting the Word Count Results
- [Exercise] Find the Total Amount Spent by Customer
- [Exercise] Check your Results, and Sort Them by Total Amount Spent
- Check Your Results and Implementation Against Mine
SparkSQL, DataFrames, and DataSets
- Introduction to SparkSQL
- [Activity] Using SparkSQL
- [Activity] Using DataFrames and DataSets
- [Exercise] Implement Friends by Age with Datasets
- Exercise Solution
- [Activity] Word Count, with Datasets
- [Activity] Minimum Temperature, with Datasets
- [Exercise] Implement Total Spent by Customer with Datasets
- Exercise Solution
Advanced Examples of Spark Programs
- [Activity] Find the Most Popular Movie
- [Activity] Use Broadcast Variables to Display Movie Names
- [Activity] Find the Most Popular Superhero in a Social Graph
- [Exercise] Find the Most Obscure Superheroes
- Exercise Solution: Find the Most Obscure Superheroes
- Superhero Degrees of Separation: Introducing Breadth-First Search
- Superhero Degrees of Separation: Accumulators, and Implementing BFS in Spark
- Superhero Degrees of Separation: Review the code, and run it!
- Item-Based Collaborative Filtering in Spark, cache(), and persist()
- [Activity] Running the Similar Movies Script using Spark’s Cluster Manager
- [Exercise] Improve the Quality of Similar Movies
Running Spark on a Cluster
- [Activity] Using spark-submit to run Spark driver scripts
- [Activity] Packaging driver scripts with SBT
- [Exercise] Use SBT and Spark-Submit Together
- Exercise Solution: Using SBT and spark-submit
- Introducing Amazon Elastic MapReduce
- Creating Similar Movies from One Million Ratings on EMR
- Partitioning
- Best Practices for Running on a Cluster
- Troubleshooting, and Managing Dependencies
Machine Learning with MLLib
- Introducing MLLib
- [Activity] Using MLLib to Produce Movie Recommendations
- Linear Regression with MLLib
- [Activity] Using Linear Regression with Spark ML
- [Exercise] Predicting Real Estate Values with Decision Trees on Spark
- Exercise Solution
Intro to Spark Streaming
- Spark Streaming with DStreams
- [Activity] Real-Time Monitoring of Most Popular Hashtags on Twitter with Spark Streaming
- Structured Streaming
- [Activity] Real-time Log Analysis with Spark Structured Streaming
- [Exercise] Windowed Operations in Structured Streaming
- Exercise Solution
Intro to GraphX
- GraphX, Pregel, and Breadth-First-Search with Pregel.
- Implementing BFS with GraphX and Pregel
- [Activity] Superhero Degrees of Separation with GraphX
You Made It! Where to Go from Here.
- Learning More, and Career Tips
- Continue your Learning Journey!

#1626 (no title)
Apache Spark with Scala – Course Materials
Apache Spark with Scala – Hands On with Big Data! Getting Started
Autonomous Cars: Deep Learning and Computer Vision in Python (Course Materials)
AWS Certified Data Analytics: Course Materials
AWS Certified Data Engineer – Hands On! Course Materials
AWS Certified Machine Learning – Course Materials
AWS Certified Machine Learning Engineer MLA-C01 Hands On! Course Materials
AWS Lambda Course
Books
Books OLD
Build a Serverless App with AWS Lambda – Hands On! Course Materials
Building Recommender Systems with Machine Learning and AI: Getting Started
Cart
Checkout
Contact Us
Course Catalog
Course Materials: Breaking Into Tech
Course Materials: Data Science and Machine Learning with Python – Hands On!
Course Materials: Deep Learning with Python, Tensorflow, and Keras – Hands On!
Course Materials: Elasticsearch and the Elastic Stack – In Depth and Hands On!
Course Materials: Machine Learning, Data Science, and Deep Learning with Python
Course Materials: The Ultimate Hands-On Hadoop
Dashboard
Data Science Course
Hadoop Ecosystem Course
Hadoop on AWS / EMR
Help
Home
Home v2
MapReduce Course
Mastering AWS Certified AI Practitioner – Hands On! Course Materials
Mastering the System Design Interview: Getting Started
Membership Catalog
My Account
Privacy Policy
Purchase
Share your success story!
Spark on AWS / EMR
Spark Streaming Course
Spark with Python Course
Spark with Scala Course
Student Directory
Taming Big Data with Apache Spark and Python – Getting Started
Taming Big Data with Spark Streaming and Scala – Getting Started
Taming Big Data with Spark Streaming and Scala – Getting Started with IntelliJ IDEA
Terms of Service
The Sundog Blog
Timeline
Under Maintenance

(C) Copyright 2021-2024 Sundog Software LLC. All rights reserved worldwide.

Theme LaunchPad by LifterLMS