Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

sdmlua/ml_fim

Open more actions menu

Repository files navigation

ml_fim

A surrogate machine learning model to improve flood depth estimation using OWP-HAND FIM depth products and other hydrological attributes.

🧪 Data Preprocessing

The process_data.py script prepares the input raster data for model training and evaluation by performing the following key steps:

✅ What It Does:

  • Reads and flattens raster data for each HUC8 watershed and return period.
  • Cleans and normalizes rasters, handling NoData values.
  • Computes derived features like aspect_sin and aspect_cos.
  • Ensures consistent shape alignment across all rasters.
  • Balances the dataset by limiting zero flood depth values to match the count of non-zero values (1:1 ratio).
  • Performs a stratified train-test split (default: 70% train, 30% test).
  • Saves:
    • One combined train.pkl file for all training data.
    • Separate *_test.pkl files for each HUC8 and return period.

⚙️ Example Usage:

python process_data.py --input ../data/ --output ../data_processed/
  • --input: Path to the base directory containing raster data folders.

  • --output: Output directory where the processed .pkl files will be saved.

  • --test_size: (Optional) Proportion of test data. Default is 0.3.

Output Structure

data_processed/
│
├── train.pkl                    # Combined, balanced training dataset
│
├── <HUC8>/
│   ├── 10year_test.pkl
│   ├── 50year_test.pkl
│   └── ...

💡 Note

  • Balancing is applied before splitting to reduce data volume and speed up training.
  • Only raster pairs with valid and aligned shapes are included.

About

A surrogate machine learning model to improve flood depth estimation using OWP-HAND FIM depth products and other hydrological attributes.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages

Morty Proxy This is a proxified and sanitized view of the page, visit original site.