Generative AI models and Applications
Faster, cheaper, better and greener AI
PyTorch MNIST Example: 16X reduction in parameter counts
nanoGPT: 27X Reduction in parameter counts
Scroll below for details
Captcha: The reverse Turing Problem:
Can AI distinguish a human from another AI?
We are all familiar with Captcha, which is used to attempt to prevent bots from submitting web forms. Typically, a user has to perform one or more tasks based on an image that is shown in order to prove that they are human. Mostly, humans find this task as unrelated and a waste of time. The advent of generative and other advanced AI techniques presents both an opportunity and a challenge. One can generate challenges on the fly that would be more interesting to humans. At the same time, bots can use similar AI technology to solve the challenge. Now, the server-side AI has to decide, based on the response received, whether it was sent by a human or by (a bot using) another AI. The real question is, "Can we generate challenges that would be easy and interesting for humans to solve and still difficult for (other) AI?" One possible answer could be multi-modal Captcha.
We are implementing the paradigm “Trust But Verify” in the form of a transformer-based fine-tuned model to model egocentric, self-preserving human behavior.
More is coming soon.
Helping Individuals and Businesses in Periodic Online Tasks
There are many tasks individuals and businesses perform online that are similar but require relevant data. We have identified a subset of those tasks that can be improved using Generative AI that can greatly benefit user experience and ease the burden of repetition. Our generative AI-based solution would save time, reduce errors and reduce user frustrations.
More is coming soon.
Improving User-Experience in the Retail Sector
Retail industry can greatly benefit from the use of new innovations and business process improvements that ultimately improve shoppers experience. We have identified a couple of key areas that are critical for improving shoppers overall experience.
More is coming soon.
Patent pending technology
AI is advancing at a rapid pace driven by domain-specific advancements and domain-independent advancements. Our core enabling technology is domain-independent and so is universally applicable.
Improved hidden layers
Lower hidden layer costs by using our proprietary normalization technology. In generative AI applications, every bit of saving counts. Lowering layer counts can allow you to increase other model parameters like the size of embeddings or the number of multiheads.
Improved model performance
Whether your output layer is regression, logistic or softmax using our patent pending technology would improve your model performance and stability.
Lower capital outlay and costs
What if you can have the same level of performance from a model at a lower capital outlay? What if you can have the same level of performance at reduced operating cost? Both are possible using our technology!
Introducing SimpLAI
Our patent-pending method reformulates the 200-year old linear regression method to use bounded linear computations. Thus, we totally avoid the vanishing gradient problem by design!
There are a few application-independent steps all leading AI models use. Most frequent ones are embedding layer, linear layer, convolutional layer, ReLU activation, dropout, attention layer, batch norm and layer norm. If a model uses batch norm, it typically won’t use layer norm and vice versa. Both batch norm and layer norm work by projecting the product of a weight vector with the input vector onto a unit ball using the mean and variance and then using affine operators to come out of the unit ball. We do not project input vectors onto a ball of any radius. Thus, we preserve the information in the relative magnitudes of the input vectors across layers.
Since the input at the input layer is bounded by design, we guarantee that inputs to all subsequent layers remain bounded in all circumstances.
The benefit of our technology is that we can lower your parameter counts, in almost all cases, in multiple ways without compromising model performance. This is just one aspect of the cost saving measures we have built into SimpLAI for you using patent-pending and proprietary methods.
SimpLAI is under continuous development. We will soon be deploying an initial version on a cloud platform.
PyTorch MNIST Results
We first took the PyTorch MNIST example and ran it as is. Then we modified it and incorporated our patent-pending technology and ran the modified version. The results are shown in the adjacent slide.
The original model uses nearly 1.2M parameters and predicts the digits of 9911 images correctly out of 10000 test images. Our method needs only about 75K parameters and it predicts the digits of 9879 images correctly out of 10000 test images.
We realize a reduction of 16X in parameter counts while maintaining almost the same level of accuracy. The reduction we can achieve depends on the initial model.
We were also able to run without any learning rate decay. We let our version run for more epoch to make sure it converges and stays converged.
We will be adding more results soon. Contact us to discuss how we can help you realize many savings using our technology. Typically, the larger your model, the more savings we can realize for you. Less total parameter count means faster inference.
NanoGPT Results
NanoGPT is intended for training/finetuning medium size GPTs and is essentially a version of GPT-2. It is an excellent package. It is a transformer-based architecture that uses layer normalization. We modified it and incorporated our patent-pending and proprietary technologies and ran both the original and our modified versions for the Shakespeare dataset. The results are shown in the adjacent slide.
The original model as mentioned at their github site uses about 0.8M parameters and produces a loss of 1.88 after 2000 epochs with a learning rate of 0.001.
We ran our version of nanoGPT with two multi-head attention layers and an embedding dimension of 32 which results in 0.03M parameters as shown in the top two screenshots on the adjacent slide. You can also see that we used a learning rate of 0.1. As shown on the bottom left screenshot our loss of 1.8862 is pretty much the same. A sample output produced by the sample.py from the nanoGPT site is shown at the bottom right screenshot. You can access the raw output here.
We realize a reduction of about 27X in parameter counts while maintaining almost the same level of accuracy. In other words, our parameter count is 3.75% of the original nanoGPT parameter count.
Our results show that we pass information better across layers as we don’t use unit ball projections, and our model converges better and has better numerical stability compared to the original nanoGPT model.
We were also able to run without any learning rate decay.
We can achieve similar results for most transformer-based architectures. Our approach is ideally suited for situations that demand continuous retraining of foundational models and for real-time and time-dependent applications.
We will be adding more results soon.
Contact us to discuss how we can help you realize many savings using our technology. Typically, the larger your model, the more savings we can realize for you. Less total parameter count means faster inference.
Ready to start?
We are a US-based company located in the Dallas-Fort Worth metroplex. In case you are considering using AI/ML for your business, we can build you a model using industry best practices. Contact us and we will send you a screenshot of how easy it is to convert the MNIST model to incorporate our technology.