(1) For using AI to code applications, see
vibe coding.
(2) Creating an AI system. Programming an AI application such as a chatbot typically means creating a neural network in Python, C++ or Java. This is the first phase in developing an AI machine learning model. Although neural network models are the basis of ChatGPT and all the chatbots that have the world's attention, there are other AI architectures (see
AI types). See
AI secret sauce.
However, for model development, AI engineers determine the number of layers and neurons in the network and the number of training passes through the network, all of which are known as "hyperparameters." From a programming perspective, the neural network becomes a multidimensional array, and this is the first phase of model development. For more details, see
neural network,
AI hyperparameter and
Tensor core.
AI Can Assist in Part or in Full
The majority of neural networks are programmed by humans who use AI libraries coded in Python or C++; thus programmers need not be mathematicians. However, to program new neural network structures that truly advance the art, or to refine existing algorithms, programmers must be proficient in algebra or calculus or both.
AI can also assist in programming some of the model code as well as refine it. In fact, AI has been known to completely code a new neural network (AI creating AI); however, there are limitations (see
neural architecture search). See
vibe coding,
neural network,
AI training and
AI programming languages.
Programming the Inference Software
Inference is the software users work with by employing the models created by the neural network to generate answers. The inference software must also be programmed, and inference programming is considered as hard core as it gets in the AI world. Programmers must be proficient not only in low-level languages and computer architecture, but in math, compilers, deep learning frameworks and hardware interfaces such as Tensor and CUDA.
After comparing regular program development with AI development in the following outline, it should be obvious that AI creation is another ballgame! See
AI glossary.
REGULAR DATA PROCESSING DEVELOPMENT:
1. design the logic
2. code the logic
3. test application
4. run application
--------------------------------------------
AI NEURAL NETWORK DEVELOPMENT:
(see AI secret sauce).
Model Development
1. INITIAL DESIGN
1a. choose model design: CNN, RNN, etc.
1b. code the model
1c. choose hyperparameters
(number of layers, neurons, passes)
1d. parameters learned by software:
(weights and biases)
2. PRE-TRAIN with data samples
(sometimes the world's information)
2a. adjust hyperparameters
2b. parameters adjusted by software
3. FINE-TUNE with correct answers
3a. hyperparameters adjusted by programmer
3b. parameters adjusted by software
3c. programmer grades results (see RLHF)
See AI hyperparameter.
Inference Development
1. design
2. code
3. optimize (see AI quantization)
Execute AI Application
1. run inference with built-in model
or
2. run inference and select model