Commits

Branch selector

 master

User selector

All users

All time

Commit History

Commits on Jul 4, 2022

• Update README.md

  HugoLaurencon

  authored
  388499d

  Copy full SHA for 388499d

Commits on Jun 13, 2022

• Update README.md

  HugoLaurencon

  authored
  d18dd77

  Copy full SHA for d18dd77

• Update README.md

  HugoLaurencon

  authored
  63edbb3

  Copy full SHA for 63edbb3

Commits on May 20, 2022

• [pre-commit.ci] pre-commit autoupdate (#415)

  Show description for 90c94e7
  pre-commit-ci[bot]

  authored
  90c94e7

  Copy full SHA for 90c94e7

• docstring not valid anymore (#417)

  HugoLaurencon

  authored
  3215a21

  Copy full SHA for 3215a21

Commits on May 11, 2022

• Update README.md

  HugoLaurencon

  authored
  e28064e

  Copy full SHA for e28064e

Commits on Apr 25, 2022

• add html lang detector (#413)

  SaulLu

  authored
  75bf926

  Copy full SHA for 75bf926

• Dedup exact lines training tokenizer dataset (#409)

  SaulLu

  authored
  534e634

  Copy full SHA for 534e634

• Add exact document deduplicate scripts (#407)

  Show description for 4560406
  

  SaulLu

  and

  thomasw21

  authored
  4560406

  Copy full SHA for 4560406

Commits on Apr 12, 2022

• Update README.md

  HugoLaurencon

  authored
  b255444

  Copy full SHA for b255444

• [pre-commit.ci] pre-commit autoupdate (#411)

  Show description for 726dbb0
  pre-commit-ci[bot]

  authored
  726dbb0

  Copy full SHA for 726dbb0

• Improved Deduplication (#412)

  Show description for 8128aa4
  

  ChenghaoMou

  and

  pre-commit-ci[bot]

  authored
  8128aa4

  Copy full SHA for 8128aa4

Commits on Mar 12, 2022

• Delete __init__.py

  huu4ontocord

  authored
  2077ca7

  Copy full SHA for 2077ca7

• Delete README.md

  huu4ontocord

  authored
  cf20a82

  Copy full SHA for cf20a82

• Delete __init__.py

  huu4ontocord

  authored
  abd6968

  Copy full SHA for abd6968

• Delete test.py

  huu4ontocord

  authored
  3023e46

  Copy full SHA for 3023e46

• Delete __init__.py

  huu4ontocord

  authored
  e7fb022

  Copy full SHA for e7fb022

• Delete memmap.py

  huu4ontocord

  authored
  eb4f77e

  Copy full SHA for eb4f77e

• Delete sql.py

  huu4ontocord

  authored
  c5fe124

  Copy full SHA for c5fe124

• Delete datastore_base.py

  huu4ontocord

  authored
  bf2a8b7

  Copy full SHA for bf2a8b7

• Delete utils.py

  huu4ontocord

  authored
  7b27e60

  Copy full SHA for 7b27e60

Commits on Mar 6, 2022

• Update person_and_id_anonymization.py

  huu4ontocord

  authored
  1a49da8

  Copy full SHA for 1a49da8

Commits on Mar 4, 2022

• Update anonymization.py

  huu4ontocord

  authored
  4e1bedb

  Copy full SHA for 4e1bedb

• Update anonymization.py

  Show description for 2905aff
  huu4ontocord

  authored
  2905aff

  Copy full SHA for 2905aff

• Create person_and_id_anonymization.py

  huu4ontocord

  authored
  db1017e

  Copy full SHA for db1017e

Commits on Mar 2, 2022

• [pre-commit.ci] pre-commit autoupdate (#410)

  Show description for b185a7a
  pre-commit-ci[bot]

  authored
  b185a7a

  Copy full SHA for b185a7a

Commits on Feb 24, 2022

• add auth token

  HugoLaurencon

  committed
  31f4597

  Copy full SHA for 31f4597

Commits on Feb 21, 2022

• Removed languages for which we don't have the filtering parameters

  HugoLaurencon

  committed
  6570774

  Copy full SHA for 6570774

• Update parameters for Basque

  HugoLaurencon

  committed
  b595a06

  Copy full SHA for b595a06

• Update parameters for Urdu

  HugoLaurencon

  committed
  4d81fa3

  Copy full SHA for 4d81fa3

• Updated parameters for Hindi

  HugoLaurencon

  committed
  c6a91d9

  Copy full SHA for c6a91d9

• Apply cleaning: remove deduplicated exact lines (#408)

  Show description for a590896
  

  3 peopleauthored
  a590896

  Copy full SHA for a590896

Commits on Feb 18, 2022

• Updated Anonymization (#406)

  Show description for b4be13f
  

  3 peopleauthored
  b4be13f

  Copy full SHA for b4be13f

Commits on Feb 17, 2022

• Update parameters for Chinese

  HugoLaurencon

  committed
  452e3ac

  Copy full SHA for 452e3ac

• Update parameters for Bengali

  HugoLaurencon

  committed
  4976e99

  Copy full SHA for 4976e99