Introduction to AI for Software Engineers: Changing the Paradigm from Logic to Probability

1. Paradigm Shift: Deterministic vs Probabilistic

• Conventional Programming:
  
  
  Data + Rules (Code/Logic) -> Output
  
   
  Example: If you want to create a fraudulent transaction detection system (fraud detection), you would write nested rules: if (transaction> IDR 50,000,000) and (location == "overseas") then flag_fraud().

• Machine Learning:
  
  
  Data + Output (Label/Answer) → Rules (Model)

2. Differentiate between AI, Machine Learning, Deep Learning, and Generative AI

+-------------------------------------------------------+
| Artificial Intelligence (AI) |
|  +-------------------------------------------------------+ |
|  | Machine Learning (ML) |  |
|  |  +---------------------------------------------+ |  |
|  |  | Deep Learning (DL) |  |  |
|  |  |  +---------------------------------------+ |  |  |
|  |  |  | Generative AI (LLM, Diffusion, etc) |  |  |  |
|  |  |  +---------------------------------------+ |  |  |
|  |  +---------------------------------------------+ |  |
|  +-------------------------------------------------------+ |
+-------------------------------------------------------+

• Artificial Intelligence (AI): Big umbrella. Any technique that allows computers to imitate human intelligence. These include ancient rule-based systems (Expert Systems), Rule-Based chess engines, to search algorithms such as $A^*$ Search.
• Machine Learning (ML): A subfield of AI in which systems learn from data without being explicitly programmed. Instead of creating a specific algorithm, we create a system that learns the algorithm itself based on historical data.
• Deep Learning (DL): A specific branch of ML that uses Artificial Neural Networks (ANN) with multiple layers (layers). Excellent at processing unstructured data (unstructured data) such as images, sounds and free text.
• Generative AI (GenAI): The part of Deep Learning focused on creating new content (text, images, audio, code) based on patterns learned from training data. The most popular examples are Large Language Models (LLM) like ChatGPT or image models like Stable Diffusion.

3. Three Main Categories of Machine Learning

A. Supervised Learning (Supervised Learning)

• Classification: Output is in the form of discrete categories.
  
  
  • Example: Predict whether an email is Spam or Not Spam.
  • Popular Algorithms: Logistic Regression, Decision Trees, Random Forest, Support Vector Machines (SVM).
• Regression: Output is a continuous value (number).
  
  
  • Example: Predict house prices based on land area, location, and number of rooms.
  • Popular Algorithms: Linear Regression, Gradient Boosting (XGBoost, LightGBM).

B. Unsupervised Learning (Unsupervised Learning)

• Clustering: Grouping data into groups based on feature similarity.
  
  
  • Example: Segmentation of e-commerce customers based on shopping behavior.
  • Popular Algorithms: K-Means, DBSCAN, Hierarchical Clustering.
• Dimensionality Reduction: Simplifies data that has thousands of columns (features) into a few main components without losing important information.
  
  
  • Example: PCA (Principal Component Analysis), t-SNE.

C. Reinforcement Learning / RL (Reinforcement Learning)

• Analogy: Teaching a dog a new trick by giving him a biscuit when he succeeds and nothing when he fails.
•  

4. The AI Development Life Cycle (Viewed from a Software Engineer's Eye)

[Problem Definition] ---> [Data Collection & Cleaning] ---> [Feature Engineering]
                                                                  |
[Deployment & MLOps] <--- [Model Evaluation] <--- [Model Training] <+

1. Data Engineering & Cleaning (80% Uptime):
   Real-world raw data is dirty: there are missing values ​​(missing values), inconsistent date formats, or outliers that ruin the analysis. Your job is to clean and standardize that data.
2. Engineering Features:
   The process of converting raw data into numerical features that are easily understood by mathematical algorithms. For example, changing the category string "Bandung City" into a vector number (One-Hot Encoding).
3. Training Model:
   The process by which an algorithm processes training data to adjust its internal weights (weights & biases). Here computing power (CPU/GPU) is used intensively.
4. Evaluation (Model Evaluation):
   Test the model on new, never-before-seen data (Test Data). The metrics used are not just "successful running or crashing", but statistical metrics such as Accuracy, Precision, Recall, F1-Score, or Mean Squared Error (MSE).
5. Deployment & MLOps:
   Wrapping the model into an accessible service (e.g. REST API using FastAPI) and integrating it into the application infrastructure.

5. Understanding Deep Learning: Anatomy of an Artificial Neural Network

Input Layer Hidden Layers Output Layer
  (x1) \ / ( ) \ /
        --> ( ) ----> ----> ( ) ---> Output (Y)
  (x2) / \ ( ) / \

Brief How Neural Networks Work:

1. Input ($X$): Incoming features (e.g. image pixels).
2. Weights ($W$) & Biases ($B$): Each inter-neuron connection has a weight. This weight is the "knowledge" or "memory" of the model.
3. Dot Product & Activation Function: Input is multiplied by weight, bias added, then passed to Activation Function (such as ReLU or Sigmoid) to provide properties non-linearity.
4. Forward Propagation: The signal moves from the input screen to the output screen until it produces a prediction.
5. Loss Function: Calculates how far the model predictions are from reality (error).
6. Backpropagation: Using calculus (Chain Rule) and the Gradient Descent to update all weights ($W$) from back to front, so that the error in the next iteration is smaller.

6. Roadmap (Roadmap) for Software Engineers

Step 1: Master the Main Language & Libraries

• Python: The de-facto standard of the AI industry. Learn basic syntax and list comprehension.
• Data Manipulation: NumPy (matrix/vector operations) and Pandas (table/DataFrame data manipulation).
• Visualization: Matplotlib and Seaborn.
• Machine Learning Traditional: Scikit-Learn (must be mastered first).
• Deep Learning: PyTorch (highly recommended for today's engineers and researchers) or TensorFlow/Keras.

Step 2: Understand Basic AI Mathematics (Intuitively)

• Linear Algebra: Vectors, Matrices, Matrix Multiplication (Dot Product), Eigenvalues. (Almost all data in AI is represented as a matrix).
• Calculus: Derivatives (Derivatives), Partial Derivatives, and Gradient Descent (how the model optimizes itself).
• Probability & Statistics: Mean, Median, Variance, Standard Deviation, Distributions, Bayes' Theorem.

Step 3: Learn MLOps and LLM Application Engineering

• LLM Engineering: Learn Prompt Engineering, Retrieval-Augmented Generation (RAG), use of Vector Databases (Chroma, Pinecone, Qdrant), as well as orchestration frameworks such as LangChain or LlamaIndex.
• MLOps: Learn how to perform versioning data (DVC), experiment tracking (MLflow, Weights & Biases), serving models (FastAPI, Triton), and monitoring model performance in production to avoid experiencing data drift.

Where to Start?

1. Project 1 (Basic ML): Create a simple house price prediction model using dataset from Kaggle with Scikit-Learn.
2.  
3. Project 3 (LLM App): Create a simple CLI or Web RAG application that can answer questions based on PDF files of your internal technical documentation.