AIPrimer.AI
  • 🚦AI Primer In Transportation
  • CHAPTER 1 - INTRODUCTION TO MACHINE LEARNING
    • Machine Learning in Transportation
    • What is Machine Learning?
    • Types of Machine Learning
      • Supervised Learning
      • Unsupervised Learning
      • Semi-supervised Learning
      • Reinforced Learning
    • Fundamental concepts of machine learning
      • Model Training and Testing
      • Evaluating the Model’s Prediction Accuracy
      • The Underfitting and Overfitting Problems
      • Bias-Variance Tradeoff in Overfitting
      • Model Validation Techniques
      • Hyperparameter Tuning
      • Model Regularization
      • The Curse of Ddimensionality
    • Machine Learning versus Statistics
  • CHAPTER 2 - SUPERVISED METHODS
    • Supervised Learning_Complete Draft
    • K-Nearest Neighbor (KNN) Algorithm
    • Tree-Based Methods
    • Boosting
    • Support Vector Machines (SVMs)
  • CHAPTER 3 - UNSUPERVISED LEARNING
    • Principal Component Analysis
      • How Does It Work?
      • Interpretation of PCA result
      • Applications in Transportation
    • CLUSTERING
      • K-MEANS
      • SPECTRAL CLUSTERING
      • Hierarchical Clustering
    • REFERENCE
  • CHAPTER 4 - NEURAL NETWORK
    • The Basic Paradigm: Multilayer Perceptron
    • Regression and Classification Problems with Neural Networks
    • Advanced Topologies
      • Modular Network
      • Coactive Neuro–Fuzzy Inference System
      • Recurrent Neural Networks
      • Jordan-Elman Network
      • Time-Lagged Feed-Forward Network
      • Deep Neural Networks
  • CHAPTER 5 - DEEP LEARNING
    • Convolutional Neural Networks
      • Introduction
      • Convolution Operation
      • Typical Layer Structure
      • Parameters and Hyperparameters
      • Summary of Key Features
      • Training of CNN
      • Transfer Learning
    • Recurrent Neural Networks
      • Introduction
      • Long Short-Term Memory Neural Network
      • Application in transportation
    • Recent Development
      • AlexNet, ZFNet, VggNet, and GoogLeNet
      • ResNet
      • U-Net: Full Convolutional Network
      • R-CNN, Fast R-CNN, and Faster R-CNN
      • Mask R-CNN
      • SSD and YOLO
      • RetinaNet
      • MobileNets
      • Deformable Convolution Networks
      • CenterNet
      • Exemplar Applications in Transportation
    • Reference
  • CHAPTER 6 - REINFORCEMENT LEARNING
    • Introduction
    • Reinforcement Learning Algorithms
    • Model-free v.s. Model-based Reinforcement Learning
    • Applications of Reinforcement Learning to Transportation and Traffic Engineering
    • REFERENCE
  • CHAPTER 7 - IMPLEMENTING ML AND COMPUTATIONAL REQUIREMENTS
    • Data Pipeline for Machine Learning
      • Introduction
      • Problem Definition
      • Data Ingestion
      • Data Preparation
      • Data Segregation
      • Model Training
      • Model Deployment
      • Performance Monitoring
    • Implementation Tools: The Machine Learning Ecosystem
      • Machine Learning Framework
      • Data Ingestion tools
      • Databases
      • Programming Languages
      • Visualization Tools
    • Cloud Computing
      • Types and Services
    • High-Performance Computing
      • Deployment on-premise vs on-cloud
      • Case Study: Data-driven approach for the implementation of Variable Speed Limit
      • Conclusion
  • CHAPTER 8 - RESOURCES
    • Mathematics and Statistics
    • Programming, languages, and software
    • Machine learning environments
    • Tools of the Trade
    • Online Learning Sites
    • Key Math Concepts
  • REFERENCES
  • IMPROVEMENT BACKLOG
Powered by GitBook
On this page
  1. CHAPTER 4 - NEURAL NETWORK

Regression and Classification Problems with Neural Networks

The NNs learn to approximate the mapping function between the pairs of input and output data using an iterative training process in a supervised learning approach. Depending on the type or the format of the output, regression and classification are the two main types of supervised learning problems. Classification refers to the problems when the output is a finite set of discrete labels or categories. For example, in the case of traffic prediction, if the prediction is in the form of explicit classes such as congested and uncongested states, the traffic prediction is a classification problem. On the other hand, regression refers to the problems when the output is numerical and continuous. When applying NN to predict the traffic state in the form of continuous numbers such as flow, density, or average speed, the prediction is considered as a regression problem.

Knowing the differences between these two types of supervised learning problems helps with the design of the output layer and selection of the loss function for the training process. For regression, the output of the network is in the form of continuous values, and the linear activation function is commonly used for the output layer. For classification, the output of the network is usually in the form of probability for each class or category. In this case, the softmax activation function is used for the last layer, which converts the output of the previous layers into probabilities that sum to one.

Different loss functions are adopted in the training of the regression and classification problems. Training of the NN is an iterative process, and the network parameters are adjusted at every step based on the performance (e.g., error) of the network estimated using the loss function. The loss function selected for classification problems for the networks with probability outputs is usually from the cross-entropy loss function family. Cross-entropy loss function enables optimizing the network parameters to return a higher probability of the correct class and lower probabilities for the other classes. For the case of regression, the loss function considers the differences between the actual and predicted values. The mean squared error is a popular loss function for the regression problems. The parameters of the network are adjusted to reduce the residuals between the prediction and the actual output variables.

PreviousThe Basic Paradigm: Multilayer PerceptronNextAdvanced Topologies

Last updated 1 year ago