Course covers big data fundamentals, processes, technologies, platform ecosystem, and management for practical application development.
-
Updated
Apr 7, 2024 - Jupyter Notebook
#
big-data-processing
Here are 82 public repositories matching this topic...
A lightweight helper utility which allows developers to do interactive pipeline development by having a unified source code for both DLT run and Non-DLT interactive notebook run.
-
Updated
Dec 7, 2022 - Python
This repository contains an Apache Flink application for real-time sales analytics built using Docker Compose to orchestrate the necessary infrastructure components, including Apache Flink, Elasticsearch, and Postgres
-
Updated
Dec 4, 2023 - Java
Eskimo is a state of the art Big Data Infrastructure and Management Web Console to build, manage and operate Big Data 2.0 Analytics clusters on Kubernetes. This is the git repository of Eskimo Community Edition.
kubernetes elasticsearch kibana kafka big-data spark bigdata kubernetes-cluster cerebro glusterfs gluster kubernetes-setup flink zeppelin webconsole big-data-platform big-data-analytics big-data-projects cluster-management big-data-processing
-
Updated
Sep 14, 2023 - Java
This code creates a Kinesis Firehose in AWS to send CloudWatch log data to S3.
big-data analytics terraform kinesis-firehose cloudwatch-logs parquet terraform-provider etl-job terraform-aws big-data-processing
-
Updated
Aug 4, 2021 - HCL
Flink SQL 实战 -中文博客专栏
-
Updated
Jun 17, 2022 - Java
Yet Another SPark Framework
-
Updated
Feb 5, 2023 - Scala
big data processing and machine learning platform，just like useing sql
-
Updated
Oct 15, 2024 - Python
GCP_Data_Enginner
python bigquery scala notebook gcp pubsub pyspark dataflow shell-script dataproc-cluster dataproc gcp-storage big-data-processing
-
Updated
Sep 4, 2021 - Shell
🛠️ Python library to import OCR data in various formats into the canonical JSON format defined by the Impresso project.
-
Updated
May 29, 2025 - Jupyter Notebook
R for Big Data (Chinese Version)
-
Updated
Nov 13, 2024 - R
Data modeling with Cassandra, building Data Warehouse using Redshift and creation of Data Lake using Spark and Airflow
-
Updated
Feb 28, 2020 - Jupyter Notebook
A curated selection of tools, libraries and services that help tame your dataflow to productively build ambitious, data driven & reactive applications on a streaming lakehouse
real-time microservices kafka big-data stream-processing big-data-analytics timetravel big-data-processing modern-data-stack elasticscaling semi-structured-cloud-warehouse
-
Updated
May 30, 2023 - Go
-
Updated
Dec 13, 2021 - Jupyter Notebook
Reservoir Sampling for Group-By Queries in Flink Platform. Answering effectively Single Aggregate.
java topic stratum apache-flink sampling reservoir-sampling streaming-data big-data-analytics group-by big-data-processing streaming-tuples
-
Updated
Aug 12, 2023 - Java
Github Repository for a versatile usable Big Data infrastructure (AVUBDI)
docker kafka spark docker-compose docker-swarm template-project big-data-platform big-data-processing
-
Updated
Feb 23, 2021 - Shell
Implementation of algorithms for big data using python, numpy, pandas.
python bloom-filter lsh streams frequent-itemset-mining pcy frequent-itemsets stream-mining shingling big-data-processing lsh-algorithm min-hasing similar-items a-priori multistage-pcy multihash-pcy
-
Updated
Apr 27, 2020 - Python
Introduction to Spark Batch processing.
-
Updated
May 27, 2024 - Scala
A pipeline that consumes twitter data to extract meaningful insights about a variety of topics using the following technologies: twitter API, Kafka, MongoDB, and Tableau.
kafka big-data mongodb twitter-api data-visualization zookeeper data-analytics kafka-consumer kafka-producer tableau nosql-database kafka-streaming big-data-processing data-processing-pipelines
-
Updated
Aug 2, 2021 - Python
Here I demonstrate the performance difference between the Poisson and the classic bootstrap by estimating the confidence interval for the difference of CTRs of the two user groups
python bootstrap statistics big-data ab-testing statistical-tests ab-tests ab-test click-through-rate big-data-processing poisson-bootstrap
-
Updated
Oct 22, 2022 - Jupyter Notebook
Improve this page
Add a description, image, and links to the big-data-processing topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the big-data-processing topic, visit your repo's landing page and select "manage topics."