Mastering Scala with Apache Spark for the Modern Data Enterprise (TTSK7520)

Duration

5 Days

30 CPD hours

About this course

This intermediate and beyond level course is geared for experienced technical professionals in various roles, such as developers, data analysts, data engineers, software engineers, and machine learning engineers.
Practical programming experience is required.

Overview

Working in a hands-on learning environment led by our expert instructor you'll:
- Develop a basic understanding of Scala and Apache Spark fundamentals, enabling you to confidently create scalable and high-performance applications.
- Learn how to process large datasets efficiently, helping you handle complex data challenges and make data-driven
decisions.
- Gain hands-on experience with real-time data streaming, allowing you to manage and analyze data as it flows into your applications.
- Acquire practical knowledge of machine learning algorithms using Spark MLlib, empowering you to create intelligent applications and uncover hidden insights.
- Master graph processing with GraphX, enabling you to analyze and visualize complex relationships in your data.
- Discover generative AI technologies using GPT with Spark and Scala, opening up new possibilities for automating content generation and enhancing data analysis.

Description

Embark on a journey to master the world of big data with our immersive course on Scala and Spark! Mastering Scala with Apache Spark for the Modern Data Enterprise is a five day hands-on course designed to provide you with the essential skills and tools to tackle complex data projects using Scala programming language and Apache Spark, a high-performance data processing engine. Mastering these technologies will enable you to perform a wide range of tasks, from data wrangling and analytics to machine learning and artificial intelligence, across various industries and applications.
Guided by our expert instructor, you'll explore the fundamentals of Scala programming and Apache Spark while gaining
valuable hands-on experience with Spark programming, RDDs, DataFrames, Spark SQL, and data sources. You'll also explore Spark Streaming, performance optimization techniques, and the integration of popular external libraries, tools, and cloud platforms like AWS, Azure, and GCP. Machine learning enthusiasts will delve into Spark MLlib, covering basics of machine learning algorithms, data preparation, feature extraction, and various techniques such as regression, classification, clustering, and recommendation systems.
You'll also gain experience working with graph processing using Spark GraphX, as well as innovative generative AI
technologies, integrating GPT with Spark and Scala for practical applications. Time permitting, you will also be introduced to Spark NLP, covering text preprocessing, classification, and sentiment analysis. With a focus on practical skills and best practices, you'll work on interesting learning objectives and gain hands-on experience with innovative tools in a live, interactive environment.
Upon completing this course, you'll be ready to confidently apply your newly acquired Scala and Apache Spark skills to a wide range of projects. You'll be able to develop efficient and scalable applications, harness the power of machine learning, and analyze large datasets, giving you a competitive edge in the rapidly evolving world of big data and analytics. By integrating these technologies into your daily work, you'll be better prepared to solve complex problems, streamline processes, and ultimately drive value for your organization.

Prerequisites

Basic understanding of Java programming: -Familiarity with Java syntax, data structures, and concepts, such as variables,
loops, and conditionals.
- Fundamental knowledge of object-oriented programming (OOP): Experience with OOP principles, such as inheritance,
encapsulation, and polymorphism, in any programming language.
- Familiarity with data structures and algorithms: A basic grasp of common data structures, such as arrays, lists, and maps,
as well as an understanding of simple algorithms, like sorting and searching.
- Experience with distributed systems: Basic awareness of distributed computing concepts, such as data partitioning,
parallel processing, and fault tolerance.
- Basic knowledge of databases: Understanding of database concepts, including data storage, querying, and manipulation
using SQL or NoSQL databases.

Introduction to Scala

Brief history and motivation
Differences between Scala and Java
Basic Scala syntax and constructs
Scala's functional programming features

Introduction to Apache Spark

Overview and history
Spark components and architecture
Spark ecosystem
Comparing Spark with other big data frameworks

Basics of Spark Programming SparkContext and SparkSession

Resilient Distributed Datasets (RDDs)
Transformations and Actions
Working with DataFrames

Spark SQL and Data Sources

Spark SQL library and its advantages
Structured and semi-structured data sources
Reading and writing data in various formats (CSV, JSON, Parquet, Avro, etc.)
Data manipulation using SQL queries

Basic RDD Operations

Creating and manipulating RDDs
Common transformations and actions on RDDs
Working with key-value data

Basic DataFrame and Dataset Operations

Creating and manipulating DataFrames and Datasets
Column operations and functions
Filtering, sorting, and aggregating data

Introduction to Spark Streaming

Overview of Spark Streaming
Discretized Stream (DStream) operations
Windowed operations and stateful processing

Performance Optimization Basics

Best practices for efficient Spark code
Broadcast variables and accumulators
Monitoring Spark applications

Integrating External Libraries and Tools, Spark Streaming

Using popular external libraries, such as Hadoop and HBase
Integrating with cloud platforms: AWS, Azure, GCP
Connecting to data storage systems: HDFS, S3, Cassandra, etc.

Introduction to Machine Learning Basics

Overview of machine learning
Supervised and unsupervised learning
Common algorithms and use cases

Introduction to Spark MLlib

Overview of Spark MLlib
MLlib's algorithms and utilities
Data preparation and feature extraction

Linear Regression and Classification

Linear regression algorithm
Logistic regression for classification
Model evaluation and performance metrics

Clustering Algorithms

Overview of clustering algorithms
K-means clustering
Model evaluation and performance metrics

Collaborative Filtering and Recommendation Systems

Overview of recommendation systems
Collaborative filtering techniques
Implementing recommendations with Spark MLlib

Introduction to Graph Processing

Overview of graph processing
Use cases and applications of graph processing
Graph representations and operations
Introduction to Spark GraphX
Overview of GraphX
Creating and transforming graphs
Graph algorithms in GraphX

Big Data Innovation! Using GPT and Generative AI Technologies with Spark and Scala

Overview of generative AI technologies
Integrating GPT with Spark and Scala
Practical applications and use cases Bonus Topics / Time Permitting

Introduction to Spark NLP

Overview of Spark NLP Preprocessing text data
Text classification and sentiment analysis

Putting It All Together

Work on a capstone project that integrates multiple aspects of the course, including data processing, machine learning, graph processing, and generative AI technologies.

Additional course details:

Nexus Humans Mastering Scala with Apache Spark for the Modern Data Enterprise (TTSK7520) training program is a workshop that presents an invigorating mix of sessions, lessons, and masterclasses meticulously crafted to propel your learning expedition forward.

This immersive bootcamp-style experience boasts interactive lectures, hands-on labs, and collaborative hackathons, all strategically designed to fortify fundamental concepts.

Guided by seasoned coaches, each session offers priceless insights and practical skills crucial for honing your expertise. Whether you're stepping into the realm of professional skills or a seasoned professional, this comprehensive course ensures you're equipped with the knowledge and prowess necessary for success.

While we feel this is the best course for the ITS Data Analytics course and one of our Top 10 we encourage you to read the course outline to make sure it is the right content for you.

Additionally, private sessions, closed classes or dedicated events are available both live online and at our training centres in Dublin and London, as well as at your offices anywhere in the UK, Ireland or across EMEA.

Training Insurance Included!

When you organise training, we understand that there is a risk that some people may fall ill, become unavailable. To mitigate the risk we include training insurance for each delegate enrolled on our public schedule, they are welcome to sit on the same Public class within 6 months at no charge, if the case arises.

What people say about us

Booking process was very easy. Once initial contact was made everything happened smoothly.

Pfizer

Great course yesterday and can’t give enough amazing feedback for John. I’ve never had a teacher hold attention of a class from start to finish, but yesterday he did this.

Jennifer O'Connor

Vistra

Alan delivered the training in an easy to understand and pleasant manner which was broken down into manageable chunks with an appropriate number of breaks to give folk time away from their laptops and screens. It is never easy to do 2 days of training on a fairly in-depth subject involving Software even in a classroom environment and remaining focused and engaged so doing it remotely adds another level of complexity, but I have to say that personally this was one of the best Training Events that I have attended for some years. I have gained so much knowledge of MS Project and it will certainly make my job easier going forward.

A large part of what I gained has to be credited to Alan’s manner and delivery and I should be grateful if you could pass on my thanks to him.

Also, the support, together with joining instructions & training literature etc from Nexus Human has been excellent – thank you for that. I intend feeding that back to our People and ICT Teams too

Nexus Human - Project class

December 2023

This was my 2nd course with Nexus Human and I enjoy the format of the courses.
This was very well-run, the instructor was superb and i will get a lot of benefit from having access for the next 6 months, I look forward to the next course.

Cathal

Kerry, Ireland

I would like to thank the staff of Nexus Human, who were very helpful, prompt, friendly and professional in helping me attain my Microsoft certification. I would like to thank Abdul in particular for his guidance during the course and commend him on his knowledge of software development.

Paul

ITC Group
Dublin

This is the 3rd course I have done with Arun. He is very helpful and always supportive if I have other questions outside the lecture parameters. I have already recommended him and Nexus Human to work colleges and would do so again after this course.

Philip

Nexus Human Classroom Student

The instructor was excellent. Was willing to field any questions we had and deal with our specific queries.

Excel student

The class just finished and I just wanted to mention to you that it was absolutely amazing. The quality of the training had very high standards, the trainer explained amazingly well and we actually practiced (did) all the steps. It was not learning by watching a presentation but by actually doing. The training has been delivered from Atlanta (US) and I cannot wait to go tomorrow in the office and recommend it. The connection and the lab environment were running perfectly.
Once again thank you very much and looking forward for the next trainings scheduled. Enrolling via Nexus Human was the perfect choice.

Cristina, Nexus Human IT Course Attendee

Firstly I would like to say thanks, I found the course I sat yesterday extremely beneficial and the instructor was fantastic, she was enthusiastic and engaging and I would highly recommend the course to friends and colleagues. The post class services are also excellent!

Adobe Student

Sharon was lovely and has a fantastic knowledge. Honestly I didn't think I needed part 1 but I am SO glad I came. Thank you. Will be back in for part 2.

Nexus Human Student

Excellent interaction and answered all of my questions. Looking forward to being able to apply some of the tips and features learned today which hopefully should have real impact on some of the more time consuming tasks currently encountered.

Nexus Human Power BI Attendee

Brilliant
Wonderful teacher
Course presented extremely well indeed

Nexus Human Student feedback

Find out more about this course

Interested in alternative dates? Would like to book a private session of this course for your company? Or for any other queries please simply fill out the form below.

Skill Up Card - Course Bundles