Replies: 6 comments · 10 replies
-
|
What does the "☁️" mean? |
Beta Was this translation helpful? Give feedback.
-
|
I don't know what the icon means, but current status of OpenCL back-end is: it works with supported models, but is buggy and perhaps, slower than it could be. |
Beta Was this translation helpful? Give feedback.
-
|
Yup, this was my impression from reading a few issues lately. If you think it's not the case, feel free to update it. I just haven't set up OpenCL in my environment and cannot do tests atm |
Beta Was this translation helpful? Give feedback.
-
|
So "Parallel decoding" is done by |
Beta Was this translation helpful? Give feedback.
-
|
Parallel decoding is also called "batched decoding" hence Naming things is hard :) Sorry if these are confusing |
Beta Was this translation helpful? Give feedback.
-
|
Should beam search be added here? I think it is broken atm, at least with CUDA. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, it should be added. The list is far from complete |
Beta Was this translation helpful? Give feedback.
-
|
Fwiw, for me beam search is broken even without CUDA in a sense that when I run the example, nothing happens (it just hangs for minutes at this line until I CTRL+C it). If it's an unknown problem, I'll open an issue (tbh, it's strange that nobody mentioned it before so maybe I'm doing something wrong). Update: when it hangs on the above mentioned line, I have 0 hard page fauls/sec. |
Beta Was this translation helpful? Give feedback.
-
|
With CUDA it works for a while, but then it starts generating gibberish. I think that the calls to |
Beta Was this translation helpful? Give feedback.
-
|
The beam search functionality should be moved out from the library and implemented as a standalone example. |
Beta Was this translation helpful? Give feedback.
-
|
What would be criteria for considering OpenCL back-end working correctly? I've fixed all known bugs in ggml-opencl.cpp and now working on refactoring like #3669. |
Beta Was this translation helpful? Give feedback.
-
|
The criteria is that if it runs correctly on your machine, then it is ✅ until someone reports a problem that is reproducible - then it becomes ☁️ or ❌ depending on how broken the thing is |
Beta Was this translation helpful? Give feedback.
-
|
Alright, turning the green light then! |
Beta Was this translation helpful? Give feedback.
-
|
maybe you can ditch the icons for something Like Scoring Like (A+, A, A-, B, ...) this will make it obvious if something working fine but needs improvements has a score with A- and so on, maybe something like this : maybe you should add a column for tier support for example, if a feature is tier 1 or 2, ... what do you think? |
Beta Was this translation helpful? Give feedback.
-
|
Is there any further progress on Finetuning on metal gpu? |
Beta Was this translation helpful? Give feedback.
-
|
This page is "[no longer updated]" since jan. 11 2024. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
[NO LONGER UPDATED]
Below is a summary of the functionality provided by the
llama.cppproject.Legend (feel free to update):
✅ - Working correctly
☁️ - Partially working
❌ - Failing
❓ - Status unknown (needs testing)
🔬 - Under investigation
🚧 - Currently in development
main,simplebatchedparallelspeculativespeculativespeculativelookaheadinfillserverembeddingmainmainmainmainmainmainmainmainmainmainmain,serverbeam-searchmaintest-tokenizer-0-llamatest-tokenizer-0-falconmainmainmainmainmainmainmainllavamainmainmainfinetunefinetuneggmlggmlggml-cudaggml-cudaggml-metalggml-openclggml-vulkanBeta Was this translation helpful? Give feedback.
All reactions