Application of our De-identification Framework with open source technologies, enabling enterprises to take ownership of the de-identification process and deploy it in trusted environments.

Este projeto é uma adaptação com base em um teste real para uma posição de Engenheiro de Dados Jr.

This repository is designed for a data science project aimed to education, wich uses a public database from brazilian educational research institute about the nationam highschool exam and applies ETL and datamining association rules to this dataset.

Acquisition of energy industry balancing and settlement calculation data, into a data lake

Azure Data engineering project

A cloud-based ETL testing and data analysis pipeline for YouTube trending video data using AWS services including Lambda, Glue, Athena, S3, and QuickSight. This project focuses on ingesting, transforming, storing, and analyzing structured and semi-structured data to generate insights based on video categories and trending metrics.

This repo is designed to show how to read and write data from/to google cloud storage with pyspark. The raw data is ingested, transformed and stored in the data lake in snapshot format.

End to end data reporting project using Azure services like Azure Data Factory for data orchestration, Azure Synapse Analytics for data warehousing, Databricks for data transformations, and Power BI for intuitive data visualization and reporting.

Building a Data Pipeline with Lakehouse Architecture on Microsoft Azure Platform