API Endpoint
The infini-gram API endpoint is https://api.infini-gram.io/
.
Please make regular HTTP POST requests.
In your request, please include a JSON payload, and the response will also contain a JSON payload.
Most queries are processed within 10s of milliseconds. You should receive the response within a fraction of a second. If you are experiencing longer latencies, it might be due to network delays or heavy traffic.
Please wrap your requests in an exception-handling loop. We can’t guarantee 100% uptime or successful processing of queries, so it may be wise to catch errors and retry the failed requests.
Getting Started
From Shell:
curl -X POST -H "Content-Type: application/json" \
-d '{"index": "v4_rpj_llama_s4", "query_type": "count", "query": "University of Washington"}' \
https://api.infini-gram.io/
Outputs:
{"count":2349754,"approx":false,"latency":0.73,"token_ids":[3014,310,7660],"tokens":["\u2581University","\u2581of","\u2581Washington"]}
From Python:
import requests
payload = {
'index': 'v4_rpj_llama_s4',
'query_type': 'count',
'query': 'University of Washington',
}
result = requests.post('https://api.infini-gram.io/', json=payload).json()
print(result)
Outputs:
{'count': 2349754, 'approx': False, 'latency': 0.724, 'token_ids': [3014, 310, 7660], 'tokens': ['▁University', '▁of', '▁Washington']}
Overview
Available indexes:
We have built the infini-gram indexes on several corpora, and you may query them through the API.
Name |
Documents |
Tokens |
Corpus |
Tokenizer |
---|---|---|---|---|
|
3,163,901,920 |
4,610,629,088,477 |
||
|
3,163,974,994 |
4,610,749,614,285 |
||
|
3,163,953,717 |
4,610,737,979,121 |
||
|
3,403,336,408 |
2,604,642,372,173 |
||
|
931,361,530 |
1,385,942,948,192 |
||
|
210,607,728 |
383,299,322,520 |
||
|
364,868,892 |
198,079,554,945 |
||
|
13,095,416 |
9,178,218,956 |
||
|
13,095,416 |
8,039,098,124 |
||
|
214,670 |
393,769,120 |
||
|
214,670 |
383,326,404 |
Input parameters:
In general, the request JSON payload should be a dict containing the following fields:
Key |
Description |
Acceptable Values |
---|---|---|
|
The index to search in |
E.g., |
|
One of the supported query types |
|
|
The query (semantic depends on query type) |
If |
For certain query types, additional fields may be required. Please see the specific query type below for more details.
If you supply a query
string, it will be tokenized into an n-gram, and we only find results that match the token boundaries. For example, doing a count query with query=``a`` will not give you the count of the letter a
, but rather the count of the token _a
.
Output parameters:
If an error occurred (e.g., malformed input, internal server error), the response JSON dict will contain a key error
with a string value describing the error.
Please check this key first before processing the rest of the response.
If the query was successful, the response JSON dict will contain the following fields:
Key |
Description |
Value Range |
---|---|---|
|
The token IDs in the tokenized query |
A list of integers |
|
The tokens in the tokenized query |
A list of strings |
|
The processing time in the engine. This does not include network latency. |
A non-negative float number, in milliseconds |
In addition, the response JSON dict will contain results specific to the query type. Please see the specific query type below for more details.
Query Types
1. Count an n-gram (or a CNF of multiple n-grams)
This query type counts the number of times the query string appears in the corpus. If the query is an empty string, the total number of tokens in the corpus will be returned.
You can simply enter a string, in which we count the number of occurrences of the string. You can also connect multiple strings with the AND/OR operators, in the CNF format, in which case we count the number of times where this logical constraint is satisfied.
Examples:
If you query
natural language processing
, the API returns the number of occurrences ofnatural language processing
.If you query
natural language processing AND deep learning
, the API returns the number of co-occurrences ofnatural language processing
anddeep learning
.If you query
natural language processing OR artificial intelligence AND deep learning OR machine learning
, the API returns the number of co-occurrences of {one ofnatural language processing
/artificial intelligence
}, and {one ofdeep learning
/machine learning
}.
Notes on CNF queries:
When you write a query in CNF, note that OR has higher precedence than AND (which is contrary to conventions in boolean algebra).
In AND queries, we can only examine co-occurrences where adjacent clauses are separated by no more than 100 tokens. You can customize this value by supplying a field
max_diff_tokens
with an integer value within range [1, 1000].In AND queries, if a clause has more than 50000 matches, the count will be approximate: we will estimate the count by examining a random subset of 50000 documents out of all documents containing that clause. You can customize this value by supplying a field
max_clause_freq
with an integer value within range [1, 500000].The above subsampling mechanism might cause a zero count on conjuction of some simple clauses (e.g.,
birds AND oil
).
Input parameters:
Key |
Description |
Acceptable Values |
---|---|---|
|
see overview |
see overview |
|
see overview |
|
|
The n-gram to count |
If |
[Optional] |
For CNF queries only. The maximum clause frequency before subsampling happens. |
An integer in range [1, 500000], default = 50000 |
[Optional] |
For CNF queries only. The maximum distance between adjacent clauses. |
An integer in range [1, 1000], default = 100 |
If you input query_ids
, it should be either a list of integers (for simple queries), or a list of list of list of integers (for CNF queries).
In case of CNF queries:
The inner-most list is a list of token IDs for a query term;
The second-level list represent a disjunctive clause, i.e., query terms connected with OR;
The outer-most list represents a CNF, i.e., disjuctive clauses connected with AND.
Here are some examples of equivalent query
and query_ids
(Assuming a Llama-2 tokenizer):
natural language processing
<==>[5613, 4086, 9068]
, or[[[5613, 4086, 9068]]]
if you willnatural language processing OR deep learning
<==>[[[5613, 4086, 9068], [6483, 6509]]]
natural language processing AND deep learning
<==>[[[5613, 4086, 9068]], [[6483, 6509]]]
natural language processing OR artificial intelligence AND deep learning OR machine learning
<==>[[[5613, 4086, 9068], [23116, 21082]], [[6483, 6509], [4933, 6509]]]
Output parameters:
Key |
Description |
Value Range |
---|---|---|
|
see overview |
see overview |
|
see overview |
see overview |
|
see overview |
see overview |
|
The number of occurrences of the query |
A non-negative integer |
|
Whether the count is approximate |
False (for exact) and True (for approximate) |
2. Prob of the last token
This query type computes the n-gram LM probability of the last token of the query conditioning on all preceding tokens. It treats your query as an n-gram, counts the full n-gram and also the (n-1)-gram that excludes the last token, and takes the division of the two counts.
Examples:
If you query
natural language processing
, the API returnsP(processing | natural language) = Cnt(natural language processing) / Cnt(natural language)
.
Notes:
If the (n-1)-gram is not found in the corpus, the returned probability will be -1.0 (to denote NaN).
Input parameters:
Key |
Description |
Acceptable Values |
---|---|---|
|
see overview |
see overview |
|
see overview |
|
|
The n-gram to query |
Any non-empty string or list of integers |
Output parameters:
Key |
Description |
Value Range |
---|---|---|
|
see overview |
see overview |
|
see overview |
see overview |
|
see overview |
see overview |
|
The n-gram LM probability |
A real number in range [0, 1], or -1.0 (denoting NaN) |
|
The count of the (n-1)-gram |
A non-negative integer |
|
The count of the n-gram |
A non-negative integer |
3. Next-token distribution
This query type treats your query as the (n-1)-gram as in query type 2, and returns the full distribution of the next token.
Examples:
If you query
natural language
, the API returnsP(* | natural language)
for all possible tokens*
.
Notes:
If the (n-1)-gram is not found in the corpus, you will get an empty distribution.
If the (n-1)-gram appears more than 1000 times in the corpus, the result will be approximate: we will estimate the distribution by examining a subset of 1000 occurrences of the (n-1)-gram. You can customize this value by supplying a field
max_support
with an integer value within range [1, 1000].
Input parameters:
Key |
Description |
Acceptable Values |
---|---|---|
|
see overview |
see overview |
|
see overview |
|
|
The (n-1)-gram to query |
Any string or list of integers (empty is OK) |
[Optional] |
The maximum (n-1)-gram frequency before subsampling happens. |
An integer in range [1, 1000], default = 1000 |
Output parameters:
Key |
Description |
Value Range |
---|---|---|
|
see overview |
see overview |
|
see overview |
see overview |
|
see overview |
see overview |
|
The count of the (n-1)-gram |
A non-negative integer |
|
The next token distribution |
A dict that maps token IDs to results. Each result is a dict with the following keys: |
|
Whether the distribution is approximate |
False (for exact) and True (for approximate) |
4. ∞-gram prob
This query type computes the ∞-gram LM probability of the last token of the query conditioning on all preceding tokens. In contrast to n-gram, the ∞-gram LM uses the longest possible (n-1)-gram suffix as context, as long as the count of this (n-1)-gram is non-zero. For more details on the ∞-gram LM, please refer to our paper.
Examples:
If you query
I love natural language processing
, andnatural language
appears in the corpus butlove natural language
does not, then the API returnsP(processing | natural language)
.
Input parameters:
Key |
Description |
Acceptable Values |
---|---|---|
|
see overview |
see overview |
|
see overview |
|
|
The sequence to query |
Any non-empty string or list of integers |
Output parameters:
Key |
Description |
Value Range |
---|---|---|
|
see overview |
see overview |
|
see overview |
see overview |
|
see overview |
see overview |
|
The n-gram LM probability |
A real number in range [0, 1] |
|
The count of the (n-1)-gram |
A non-negative integer |
|
The count of the n-gram |
A non-negative integer |
|
The number of tokens in the longest suffix used to compute the ∞-gram probability |
A non-negative integer |
|
The longest suffix used to compute the ∞-gram probability |
A string (may be empty) |
5. ∞-gram next-token distribution
This query type computes the full next-token distribution according to the ∞-gram LM. It uses the longest possible (n-1)-gram suffix of the query as context, as long as the count of this (n-1)-gram is non-zero. For more details on the ∞-gram LM, please refer to our paper.
Examples:
If you query
I love natural language
, andnatural language
appears in the corpus butlove natural language
does not, then the API returnsP(* | natural language)
for all possible tokens*
.
Input parameters:
Key |
Description |
Acceptable Values |
---|---|---|
|
see overview |
see overview |
|
see overview |
|
|
The sequence to query |
Any string or list of integers (empty is OK) |
[Optional] |
The maximum (n-1)-gram frequency before subsampling happens. |
An integer in range [1, 1000], default = 1000 |
Output parameters:
Key |
Description |
Value Range |
---|---|---|
|
see overview |
see overview |
|
see overview |
see overview |
|
see overview |
see overview |
|
The count of the (n-1)-gram (where (n-1) is the number of tokens in the longest suffix) |
A non-negative integer |
|
The next token distribution |
A dict that maps token IDs to results. Each result is a dict with the following keys: |
|
Whether the distribution is approximate |
False (for exact) and True (for approximate) |
|
The number of tokens in the longest suffix used to compute the ∞-gram probability |
A non-negative integer |
|
The longest suffix used to compute the ∞-gram probability |
A string (may be empty) |
6. Search documents
This query type returns documents in the corpus that match your query.
6.1 Search with simple queries
With simple queries, the engine can return documents containing a single n-gram.
Examples:
If you query
natural language processing
, the documents returned would contain the stringnatural language processing
.
Step 1: find
First, you need to make a find
query to get information about where the matching documents are located.
Input parameters:
Key |
Description |
Acceptable Values |
---|---|---|
|
see overview |
see overview |
|
see overview |
|
|
The search query |
If |
Output parameters:
Key |
Description |
Value Range |
---|---|---|
|
see overview |
see overview |
|
The number of occurrences of the query |
A non-negative integer |
|
The segment of each suffix array shard that matches the query |
A list of 2-tuples, each tuple is a pair of non-negative integers, where the second integer is no smaller than the first integer |
The returned segment_by_shard
is a list of 2-tuples, each tuple represents a range of “ranks” in one of the shards of the index, and each rank can be traced back to a matched document in that shard.
The length of this list is equal to the total number of shards.
Step 2: get_doc_by_rank
Then, you can use the get_doc_by_rank
query to retrieve a matching document by any rank in the segment.
Input parameters:
Key |
Description |
Acceptable Values |
---|---|---|
|
see overview |
see overview |
|
see overview |
|
|
The shard index |
An integer in range [0, |
|
A rank in the shard |
An integer in range [ |
[Optional] |
The maximum number of tokens to return |
An integer in range [1, 10000], default = 1000 |
For example, if you want to retrieve the first matched document in shard 0, you should make the query with s=0
and rank=segment_by_shard[0][0]
.
Output parameters:
Key |
Description |
Value Range |
---|---|---|
|
see overview |
see overview |
|
The index of the document in the corpus |
A non-negative integer |
|
The total number of tokens in the document |
A non-negative integer |
|
The number of tokens returned after truncation |
A non-negative integer |
|
The token IDs in the tokenized document |
A list of integers |
|
The spans of the document |
A list of tuples, each tuple is a pair of a span of text and a string marking the index of the clause that this span matches; if this span does not match any clause, this second element is NULL |
6.2 Search with CNF queries
You can also connect multiple strings with the AND/OR operators, in the CNF format, in which case the returned documents contains strings such that it satisfies this logical constraint.
Examples:
If you query
natural language processing AND deep learning
, the documents returned would contain bothnatural language processing
anddeep learning
.If you query
natural language processing OR artificial intelligence AND deep learning OR machine learning
, the documents returned would contain at least one ofnatural language processing
/artificial intelligence
, and also at least one ofdeep learning
/machine learning
.
Notes on CNF queries:
When you write a query in CNF, note that OR has higher precedence than AND (which is contrary to conventions in boolean algebra).
In AND queries, we can only examine co-occurrences where adjacent clauses are separated by no more than 100 tokens. You can customize this value by supplying a field
max_diff_tokens
with an integer value within range [1, 1000].In AND queries, if a clause has more than 50000 matches, the count will be approximate: we will estimate the count by examining a random subset of 50000 documents out of all documents containing that clause. You can customize this value by supplying a field
max_clause_freq
with an integer value within range [1, 500000].The above subsampling mechanism might cause a zero count on conjuction of some simple clauses (e.g.,
birds AND oil
).
Step 1: find_cnf
First, you need to make a find_cnf
query to get information about where the matching documents are located.
Input parameters:
Key |
Description |
Acceptable Values |
---|---|---|
|
see overview |
see overview |
|
see overview |
|
|
The search query |
If |
[Optional] |
For CNF queries only. The maximum clause frequency before subsampling happens. |
An integer in range [1, 500000], default = 50000 |
[Optional] |
For CNF queries only. The maximum distance between adjacent clauses. |
An integer in range [1, 1000], default = 100 |
If you input query_ids
, it should be a list of list of list of integers.
The inner-most list is a list of token IDs for a query term;
The second-level list represent a disjunctive clause, i.e., query terms connected with OR;
The outer-most list represents a CNF, i.e., disjuctive clauses connected with AND.
Here are some examples of equivalent query
and query_ids
(Assuming a Llama-2 tokenizer):
natural language processing OR deep learning
<==>[[[5613, 4086, 9068], [6483, 6509]]]
natural language processing AND deep learning
<==>[[[5613, 4086, 9068]], [[6483, 6509]]]
natural language processing OR artificial intelligence AND deep learning OR machine learning
<==>[[[5613, 4086, 9068], [23116, 21082]], [[6483, 6509], [4933, 6509]]]
Output parameters:
Key |
Description |
Value Range |
---|---|---|
|
The token IDs in the tokenized query |
A list of integers, or a triply-nested list of integers |
|
The tokens in the tokenized query |
A list of strings, or a triply-nested list of strings |
|
see overview |
see overview |
|
The number of occurrences of the query |
A non-negative integer |
|
Whether the count is approximate |
False (for exact) and True (for approximate) |
|
The pointers to the matching documents in each shard |
A list of lists of non-negative integers |
Step 2: get_doc_by_ptr
Then, you can use the get_doc_by_ptr
query to retrieve a matching document by any pointer in the segment.
Input parameters:
Key |
Description |
Acceptable Values |
---|---|---|
|
see overview |
see overview |
|
see overview |
|
|
The shard index |
An integer in range [0, |
|
The pointer to the document |
An integer in |
[Optional] |
The maximum number of tokens to return |
An integer in range [1, 10000], default = 1000 |
For example, if you want to retrieve the first matched document in shard 0, you should make the query with s=0
and ptr=ptrs_by_shard[0][0]
.
Output parameters:
Key |
Description |
Value Range |
---|---|---|
|
see overview |
see overview |
|
The index of the document in the corpus |
A non-negative integer |
|
The total number of tokens in the document |
A non-negative integer |
|
The number of tokens returned after truncation |
A non-negative integer |
|
The token IDs in the tokenized document |
A list of integers |
|
The spans of the document |
A list of tuples, each tuple is a pair of a span of text and a string marking the index of the clause that this span matches; if this span does not match any clause, this second element is NULL |