Comprehensive Rust 馃 translation of the book's code! #474
Replies: 2 comments · 10 replies
-
|
Glad you enjoyed the book. And wow, your repo is an amazing project! Haven't dabbled with Rust, yet, but now there's one more reason to get into it 馃槉. Thanks for sharing!! |
Beta Was this translation helpful? Give feedback.
-
|
馃挴 on your note dev vs. prod So the rust translation is also able to run on cuda-enabled devices! (Candle also supports metal backend, but I haven't tested out Metal in my project). # with cuda
cargo run --features cuda example 05.07
# without cuda
cargo run example 05.07If you try and run with the |
Beta Was this translation helpful? Give feedback.
-
|
Oh that's great! Btw regarding Metal, it works pretty well now as of a few months ago, but it you'll still see some numerical discrepancies vs CUDA or CPU. This becomes especially apparent during training runs where small rounding/precision differences add up over time. During inference, there's a non-noticeable difference though. |
Beta Was this translation helpful? Give feedback.
-
|
Oh interesting! Thanks for this information. In that case, I should probably test these translations on Metal. |
Beta Was this translation helpful? Give feedback.
-
|
I must say it got better in recent months though. And it mostly only affected training where numerical differences add up during backprop. But yeah, here are some relevant discussions/observations from the Manning reader forum (you may have to be logged in to see the full reader comments; if you got the book from Amazon, there should be instructions on the first page on how to connect the book to the Manning account): |
Beta Was this translation helpful? Give feedback.
-
|
Oh this is great! I'll give this a look. Thanks |
Beta Was this translation helpful? Give feedback.
-
|
Related FWIW. Huggingface's TGI is written in Rust: https://huggingface.co/docs/text-generation-inference/en/index |
Beta Was this translation helpful? Give feedback.
-
|
Whoa, I had no idea! Thanks for sharing! |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hey all,
First off, would like to say this is probably near the top, if not at the top, of my favourite DL books to read. I personally appreciated the careful explanations in every aspect of building an LLM -- I learned a lot in reading this book.
Second, I'm just about finished translating nearly all of the book's code into Rust using the Candle ML/DL framework. So, I thought it'd be a good time to share here in case there are others that might be interested in following this fantastic text with Rust "equivalent" code. Link to the repo and the docs are provided below. Contributions are welcome 馃
Cheers!
mainbranch of the github repo.)Beta Was this translation helpful? Give feedback.
All reactions