quote [ tinygrad

We write and maintain tinygrad, the fastest growing neural network framework (over 23,000 GitHub stars)

It's extremely simple, and breaks down the most complex networks into 3 OpTypes
ElementwiseOps are UnaryOps, BinaryOps, and TernaryOps.
They operate on 1-3 tensors and run elementwise.
example: SQRT, LOG2, ADD, MUL, WHERE, etc...

ReduceOps operate on one tensor and return a smaller tensor.
example: SUM, MAX

MovementOps are virtual ops that operate on one tensor and move the data around
Copy-free with ShapeTracker.
example: RESHAPE, PERMUTE, EXPAND, etc... ]