https://kinianlo.github.io/tags/llm/ Recent content in LLM on Kin Ian Lo https://kinianlo.github.io/images/papermod-cover.png https://kinianlo.github.io/images/papermod-cover.png Hugo -- 0.127.0 en Mon, 17 Jun 2024 01:06:04 +0000 https://kinianlo.github.io/posts/llm-api-probs/ Mon, 17 Jun 2024 01:06:04 +0000 https://kinianlo.github.io/posts/llm-api-probs/ OpenAI has modified their API to return the log probabilities before any logit bias is applied. Hence the methods described in this article are no longer applicable. The aim of this article is to provide an alternative derivation of the results in Mattew Finlayson’s article Obtaining logprobs from an LLM API. Most LLM APIs return the logprobs of only the top-$ k $ predictions. The value of $ k $ is often small, in the order of 10.