by dluc · Agent Tool · ★ 101
GPT Tokenizer .NET / C# When using OpenAI GPT, you may need to know how many tokens your code is using for various purposes, such as estimating costs and improving results. The C# class can help you count tokens in your prompts and in the responses received. The tokenizer uses a byte-pair encoding (BPE) algorithm to split words into subwords based on frequency and merges rules. It can handle out-of-vocabulary words, punctuation, and special tokens. The result of this library is compatible with OpenAI GPT tokenizer that you can also test here.
| Stars | 101 |
| Forks | 14 |
| Language | C# |
| Category | Agent Tool |
| License | CC0-1.0 |
| Quality Score | 63.6020376912277/100 |
| Open Issues | 5 |
| Last Updated | 2023-05-04 |
| Created | 2022-10-17 |
| Platforms | dotnet |
| Est. Tokens | ~48k |
Explore other popular agent tool tools:
openai-tools is A collection of tools for working with OpenAI. It is categorized as a Agent Tool with 101 GitHub stars.
openai-tools is primarily written in C#. It covers topics such as gpt-3, gpt3, openai.
You can find installation instructions and usage details in the openai-tools GitHub repository at github.com/dluc/openai-tools. The project has 101 stars and 14 forks, indicating an active community.
openai-tools is released under the CC0-1.0 license, making it free to use and modify according to the license terms.