Feather, On Air.

Compare performance benchmarks on the Feather API.

AI Research

Evals

TL;DR

  • We provide browser and API access to Feather 1.7B, Qwen3 1.7B, Qwen3 4B, Qwen3 8B, and Qwen2.5-Math 1.5B, with direct comparisons on the same prompt.
  • The playground includes 68 archived prompt templates from 28 generation benchmarks used in our evals.
  • The original evaluation prompts can be used as written, modified, or replaced with custom prompts.
  • All five models are base models i.e. not instruction tuned or RL'ed.

Note: Models at the 1.7B scale are not cleanly reliable and should only be treated as proxies for scaled model capabilities.

Find Benchmarks

Prefill benchmark prompts or write your own.

Benchmark

Compare models

Compare Feather against Qwen.

Models to compare
Prompt format
Max output

Recorded responses

Recorded example responses · 256 tokens max. Compare to generate fresh answers.

Feather 1.7B

Consider the function
f(x)={x2sin⁡(1/x)x≠00x=0f(x)=\begin{cases}x^2\sin(1/x)&x\ne0\\0&x=0\end{cases}
This function has a derivative of zero at x=0x=0, but it is not a maximum or minimum at x=0x=0. This is because the function oscillates between positive and negative values as xx approaches 0, and the derivative changes sign at x=0x=0.

Qwen3 1.7B

Yes, a function can have a derivative zero at a point without having a maximum or minimum there. Consider the function f(x)=x3f(x) = x^3 at x=0x = 0. The derivative of f(x)f(x) is f′(x)=3x2f'(x) = 3x^2, which is zero at x=0x = 0. However, x=0x = 0 is not a maximum or minimum point for f(x)f(x), as the function is increasing on both sides of x=0x = 0.

Qwen2.5 Math 1.5B

Yes, a function can have a derivative of zero at a point without having a maximum or minimum there. Consider the function f(x)=x3f(x) = x^3. The derivative of this function is f′(x)=3x2f'(x) = 3x^2, which equals zero at x=0x = 0. However, f(x)=x3f(x) = x^3 does not have a maximum or minimum at x=0x = 0 because the function is increasing on both sides of x=0x = 0. This is because the derivative f′(x)=3x2f'(x) = 3x^2 is always non-negative, indicating that the function is always increasing or constant.

Qwen3 4B

Yes. Consider the function f(x) = x^3. The derivative of this function is f'(x) = 3x^2. At x = 0, the derivative is zero, but the function does not have a maximum or minimum there.

Qwen3 8B

Yes. Consider the function f(x) = x^3. It has derivative zero at x = 0, but it has neither a maximum nor a minimum there.

Use the API

Query Feather and the Qwen models from your own code.

Python

import json, os, urllib.request

payload = {
    "model": "feather",
    "prompt": "Explain the difference between a derivative and an integral.",
    "format": "qa",
    "max_tokens": 256,
}
request = urllib.request.Request(
    os.environ["HYPERSTITION_URL"].rstrip("/") + "/api/benchmark",
    data=json.dumps(payload).encode("utf-8"),
    headers={
        "Authorization": "Bearer " + os.environ["HYPERSTITION_API_KEY"],
        "Content-Type": "application/json",
    },
)
with urllib.request.urlopen(request, timeout=600) as response:
    for line in response:
        event = json.loads(line)
        if event["type"] == "delta":
            print(event["text"], end="", flush=True)
        elif event["type"] == "error":
            raise RuntimeError(event.get("error", "Generation failed"))
print()

curl

curl -N "$HYPERSTITION_URL/api/benchmark" \
  -H "Authorization: Bearer $HYPERSTITION_API_KEY" \
  --json @- <<'JSON'
{
  "model": "feather",
  "prompt": "Explain the difference between a derivative and an integral.",
  "format": "qa",
  "max_tokens": 256
}
JSON

More like this