You need to log in to make submissions.
Please read the general instructions for this exercise first. Here are the additional instructions specific to this task:
Implement efficient processing of an LLM prompt using all available single-precision processing power of the CPU.
I will first run all kinds of tests to see that your code works correctly. You can try it out locally by running ./grading test, but please note that your code has to compile and work correctly not only on your own computer but also on our machines.
If all is fine, I will run the benchmarks. You can try it out on your own computer by running ./grading benchmark, but of course the precise running time on your own computer might be different from the performance on our grading hardware.
| Name | Parameters |
|---|---|
| benchmarks/1 | dim = 288, n_heads = 6, n_layers = 6, num_tokens = 127, vocab_size = 2048 |
| Process a sequence of 127 tokens with a 6 layer model of width 288 | |
| benchmarks/2 | dim = 512, n_heads = 8, n_layers = 8, num_tokens = 765, vocab_size = 2048 |
| Process a sequence of 765 tokens with a 8 layer model of width 512 | |
| benchmarks/3 | dim = 768, n_heads = 12, n_layers = 6, num_tokens = 510, vocab_size = 32000 |
| Process a sequence of 510 tokens with a 6 layer model of width 768 | |
| benchmarks/4 | dim = 288, n_heads = 6, n_layers = 6, num_tokens = 2047, vocab_size = 2048 |
| Process a sequence of 2047 tokens with a 6 layer model of width 288 | |
| benchmarks/5 | dim = 768, n_heads = 12, n_layers = 12, num_tokens = 513, vocab_size = 32000 |
| Process a sequence of 513 tokens with a 12 layer model of width 768 | |
In this task your submission will be graded using benchmarks/5: Process a sequence of 513 tokens with a 12 layer model of width 768.
The point thresholds are as follows. If you submit your solution no later than on Wednesday, 28 August 2024, at 23:59:59 (Helsinki), your score will be:
| Running time | Points |
|---|---|
| ≤ 8.000 sec | 1 |
| ≤ 6.000 sec | 2 |
| ≤ 4.000 sec | 3 |
| ≤ 2.000 sec | 4 |
| ≤ 1.000 sec | 5 |
For late submissions you will not get any points.
Your submissions to this task will also automatically take part in the contest, and you can receive up to 2 additional points if your code is among the fastest solutions this year!
| Running time | Extra points |
|---|---|
| ≤ 1.20 × fastest | 1 |
| ≤ 1.05 × fastest | 2 |