📖 llm-tracker

Home

❯

logbook

❯

JA Dataset Cleaning

JA Dataset Cleaning

Jan 09, 20241 min read

See also: https://github.com/AUGMXNT/shisa/wiki/shisa%E2%80%90v2

Setup Lilac

  • https://github.com/lilacai/lilac
  • https://www.lilacml.com/

Data cleanup

  • https://huggingface.co/datasets/augmxnt/ultra-orca-boros-en-ja-v1
  • https://huggingface.co/datasets/augmxnt/shisa-en-ja-dpo-v1
  • https://huggingface.co/datasets/NTQAI/sharegpt-clean-ja

Bagel to JA?

  • https://github.com/jondurbin/bagel
  • https://huggingface.co/jondurbin/bagel-8x7b-v0.2
  • https://huggingface.co/jondurbin/bagel-34b-v0.2

  • Setup Lilac
  • Data cleanup
  • Bagel to JA?

Created with Quartz © 2026