Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

llama : llama_perf + option to disable timings during decode #9355

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Sep 13, 2024
Merged

llama : llama_perf + option to disable timings during decode #9355

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
471e7e1
llama : llama_perf + option to disable timings during decode
ggerganov Sep 7, 2024
ade52b6
common : add llama_arg
ggerganov Sep 8, 2024
6cce78c
Merge branch 'master' into gg/llama-perf
ggerganov Sep 10, 2024
fd46535
Update src/llama.cpp
ggerganov Sep 10, 2024
f42de24
perf : separate functions in the API
ggerganov Sep 11, 2024
7362f28
perf : safer pointer handling + naming update
ggerganov Sep 12, 2024
44f0218
Merge branch 'master' into gg/llama-perf
ggerganov Sep 12, 2024
f35e9b8
minor : better local var name
ggerganov Sep 12, 2024
444b757
perf : abort on invalid sampler pointer
ggerganov Sep 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
minor : better local var name
  • Loading branch information
@ggerganov
ggerganov committed Sep 12, 2024
commit f35e9b87cd259cbe9fa618ac9125f040ebc3e4b8
10 changes: 5 additions & 5 deletions 10 src/llama-sampling.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1667,10 +1667,10 @@ struct llama_perf_sampler_data llama_perf_sampler(const struct llama_sampler * c
return data;
}

const auto * p = (const struct llama_sampler_chain *) chain->ctx;
const auto * ctx = (const struct llama_sampler_chain *) chain->ctx;

data.t_sample_ms = 1e-3 * p->t_sample_us;
data.n_sample = std::max(0, p->n_sample);
data.t_sample_ms = 1e-3 * ctx->t_sample_us;
data.n_sample = std::max(0, ctx->n_sample);

return data;
}
Expand All @@ -1688,7 +1688,7 @@ void llama_perf_sampler_reset(struct llama_sampler * chain) {
return;
}

auto * p = (struct llama_sampler_chain *) chain->ctx;
auto * ctx = (struct llama_sampler_chain *) chain->ctx;

p->t_sample_us = p->n_sample = 0;
ctx->t_sample_us = ctx->n_sample = 0;
}
Loading
Morty Proxy This is a proxified and sanitized view of the page, visit original site.