There are several ways to go around building an AI application, this article focuses on building web applications with integrated AI features. With a concentration on architectural patterns. To make your life easier, it is always better to know a few of these architectures and know when to use them. This further shifts your mind to pick a good stack depending on the type of architecture you select to build your application. Most developers miss these steps and fall for the trap that comes with the euphoria for building applications like these. Don't get me wrong, building needs to be fun, building in itself is a euphoric process but that is going to be dependent on you. Imagine typing your brains out, getting all the steps right, no type errors, no deprecated libraries; just straight “GOD LEVEL” development flow, then suddenly you realize superbase's pg_vector extension is good, but requires you to find another way to get your vectors into dimensions suitable or what the database row should be receiving. You then scour the internet looking for a suitable embedding library only to find out this is going to be more stressful than you envisioned it to be. I am a stern believer in the whole learning process where you try something, fail at it and get better by learning from your mistakes even when it's a slow process. But at certain times, certain tools are just best for a specific type of job. In this scenario, pinecone is a far better alternative to superbase when it comes to storing vector data , it comes with a free inbuilt embeddings model and it's super easy to implement.

Passo Uno (Understanding the terms)

If you already got lost with the terms used above. These are a few you’d have to note down in order to understand the rest that are going to appear in this series. 

Embedings 

An embedding is a representation of images, texts, audios, videos into values that can be used by machine learning models and semantic search algorithms. Machine Learning models need values to be in a mathematical form in order for them to be understood. Embeddings allow machine learning models to find similar objects based on factors used in creating the embeddings. 

Vector

A vector is a numeric representation of an object, it can be used to represent anything , texts, images, videos, pixel values, model weights, random points in space. Remember, a vector has no meaning by itself. It is simply a mathematical object. 

Vector Databases

Vector databases are a special kind of database designed to store and search up vector embeddings efficiently, especially for tasks like semantic searches, recommendations and AI retrieval. Unlike most common databases used for building applications, vector databases are more cut out for the task when it comes to dealing with machine learning models, and in general anything that has to do with AI. Models don’t understand regular tasks, images , audio etc. This doesn’t mean that SQL databases don’t play a role in building AI web applications, because they do. But when it comes to building models, vectors are a must.

Neutral Networks

We’re not going to use this in the series, but as a bonus; it’ll be good to have in your play book. A neural network is a model made up of many small units called neurons, arranged in layers. They are a type of machine learning mode inspired by how the human brain works. Each neuron takes in numbers (vectors) , makes calculations on them and outputs vectors.

RAG (Retrieval-Augmented Generation)

When it comes to RAGs, always think about giving AI access to a knowledge base or library so it can look up answers and use them as reference points before giving a response. For now I'm going to keep it simple. As we move along we would develop more on this understanding.

WASM (WebAssembly)

WebAssembly lets you run languages like C, C++ , Rust, Go, Python and even ML runtimes inside the browser at near-native speed. It is a compact file that browsers can execute efficiently. This is getting essential in the ML verse because now we ONNX Runtime right in the browser with no GPU and still get awesome speed. Train your ML’s with your GPU, export to ONNX runtime and run in WASM.And not just on browsers, WASM can run on server so it's very efficient for NextJS. So when you want to build something that runs locally, that does not depend on the internet, full control, no API cost, no network latency,  WASM is the way to go. The only pitfall for WASM is when the AI model is very large. 

ONNX (Open Neural Network Exchange) Runtime

ONNX runtime is a high performance inference engineer designed to run machine learning models quickly and efficiently across many platforms (web , mobile ,server, edge, devices). It is framework agnostic , meaning regardless of whatever framework the machine learning model was trained or built in, whether PyTorch, TensorFlow, Keras, Scikit-learn or XGBoost . The ONNX runtime uses optimized kernels to run models faster than running them directly from the framework it was made in. And since it's agnostic, it’s able to make the models portable to run on NodeJs , WebAssembly (browser offline inference), Python, C++, Mobile (Android/IOS), Edge Devices (Jetson, Raspberry Pi etc.) This is very important to understand because it would be used in one of the architectures in this series.

Runtime

A runtime is the environment where your code or model actually runs. It loads the code up, manages it while it runs, gives it access to resources like memory, CPU, files etc. And for our series we’d be looking more into the ML runtime (ONNX runtime) and the JavaScript Runtime used in the browser, with providers like fetch API, fs, timers, event loops etc.

Edge Runtime

Just like any runtime, the edge runtime is a lightweight environment designed to run your code physically closer to the user, at the edge of their network , instead of a centralized server. Unlike what i talked about under the ONNX Runtime Web which operates within the regular NodeJs runtime, the edge runtime uses resources and runs your applications from the nearest node to you.