My attempt to make something like TinyGrad or PyTorch
A framework like PyTorch & MicroGrad written fully in python(i will add the c & cpp components for faster implementation though).
It's supposed to be a good and lightweight C and Python based deep learning framework, which it's not, as of now(still building).
It contains a framework similar to Numpy which allows to do basic matrix operations like element-wise add/mul + matrix multiplication + broadcasting. Also building pytorch like auto-differentiation engine: axgrad (work in progress!)
It has basic building blocks required to build a neural network:
- Basic tensor ops framework that could easily so matrix add/mul (element-wise), transpose, broadcasting, matmul, etc.
- A gradient engine that could compute and update gradients, automatically, much like micrograd, but on a tensor level ~ autograd like (work in progress!).
- Optimizer & loss computation blocks to compute and optimize (work in progress!). i'll be adding more things in future...
Development | Status | Feature |
---|---|---|
Base Class | in progress |
|
Versions | in progress |
|
Loss | in progress |
|
Language Transformer | in progress |
|
Convolutional Neural Network | in progress |
|
Neural Network Components | in progress |
|
This shows basic usage of axgrad.engine
& few of the axon
's modules to preform tensor operations and build a sample neural network
anyway, prefer documentation for detailed usage guide:
- axon.doc: for using like numpy
- axgrad.doc: for building neural network from axon library (incomplete for now)
It's similar to NumPy, for now. I'm trying to add more functions/methods to make it equivalent to PyTorch, at-least to some extent. It supports a few basic functions for now, like element wise ops: add/sub/mul/div/pow; tensor ops: matmul, flatten, 2d-convolution, transpose, shape, etc.
from axon import tensor
# initializing 2-d matrices
x = tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
y = tensor([[9, 8, -7], [6, -5, 4], [3, -2, -1]])
z = x + y # addition
print(z) # output: axon.tensor([10, 10, -4], [10, 0, 10], [10, 6, 8])
import axon
zeros = axon.zeros([1, 4, 5]) # 3-d matrix containg zeros, (float)
ones = axgon.ones([3, 4]) # 2-d matrix containing ones, (int)
x = tensor([[1, 4, 4], [1, 5, 6], [1, 5, 7]])
z = tensor([[1, 1, 1, 1], [4, 5, 5, 4], [4, 6, 7, 5]])
print(tensor.matmul(x, z)) # out: axon.tensor([33, 45, 49, 37], [45, 62, 68, 51], [49, 68, 75, 56])
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change. Please make sure to update tests as appropriate. But it's still a work in progress.
None!