Artificial Intelligence Course

Natural Language Processing
Part I NLTK

• What is NLP?
• Typical NLP Tasks
• Morphology
• Sentence Segmentation & Tokenization
• Pattern Matching with Regular Expression
• Stemming, Lemmatization
• Stop Words Removal (English)
• Corpora/Corpus
• Context Window – Bigram, Ngram
• Applications of NLP
• Introduction to the NLTK Library
• Processing Raw Text
• Regular Expression
• Normalising Text
• Processing Raw Text – Tokenise Sentences
• String Processing with Regular Expression, Normalising Text
• Extracting Features from Text
• Bag-of-Words(BoW), TF-IDF
• Similarity score Cosine similarity
• Computer Vision
• Image Formation
• Sampling and Quantisation
• Image Processing – flipping, cropping, rotating, scaling
• Image statistics & Histogram
• Spatial Resolution
• Gray level/Intensity Resolution
• Spatial Filtering
• Convolution
• Smoothing, Sharpening
• Color Space Conversion &
• Histogram
• Thresholding for Binarization
• Morphological Operations
• Image Gradient
• Bounding Box
• Sobel’s Edge Detection Operator
• Template Matching
• Image Feature – Keypoint and Descriptor
• Harris Corner Detector
• Object Detection with HoG
• Stream Video Processing with OpenCV

• Advance NLP
• • “Use Logistic Regression, 
  • Naive Bayes and Word vectors to implement Sentiment Analysis”
  • R-CNN
  • RNN
  • Encoder-Decoder
  • Transformer
  • Reformer
  • Embeddings
  • Information Extraction
  • LSTM
  • Attention
  • Named Entity Recognition
  • Transformers
  • HuggingFace
  • BERT
  • Text Generation
  • Named Entity Recognition
  • GRU
  • Siamese Network in TensorFlow
  • Self Attention Model
  • Advanced Machine Translation of Complete Sentences
  • Text Summarization

• Prompt Engineerin
• Why Prompt Engineering?
• ChatGPT
• Few Standard Definitions:
• Label, Logic
• Model Parameters (LLM Parameters)
• Basic Prompts and Prompt Formatting
• Elements of a Prompt, Context
• Task Specification
• Constraints
• General Tips for Designing Prompts:
• Be Specific ,Keep it Concise
• Be Contextually Aware
• Test and Iterate
• Prompt Engineering Use Cases
• Information Extraction
• Text Summarization
• Question Answering
• Code Generation
• Text Classification
• Prompt Engineering Techniques
• N-shot Prompting
• Zero-shot Prompting
• Chain-of-Thought (CoT) Prompting
• Generated Knowledge Prompting

• Problem Identification and Definition
  • Define a clear problem statement or task that the AI project aims to address. This could involve tasks such as image classification, natural language processing, predictive analytics, etc.
  • Understand the scope, objectives, and constraints of the project.
• Data Collection and Preprocessing
  • Identify relevant data sources needed to train and evaluate the AI model.
  • Clean and preprocess the data to handle missing values, outliers, and inconsistencies.
  • Perform exploratory data analysis (EDA) to understand the characteristics and distribution of the data.
• Feature Engineering
  • Extract and select appropriate features from the data that are relevant to the problem statement.
  • Transform and engineer features to enhance the performance of the AI model.
• Model Selection
  • Choose suitable AI models based on the nature of the problem and data.
  • Experiment with different algorithms such as supervised (e.g., regression, classification), unsupervised (e.g., clustering), or reinforcement learning based on project requirements.
• Model Training and Evaluation
  • Split the dataset into training, validation, and test sets.
  • Train the selected AI model using the training data and validate its performance using the validation set.
  • Evaluate the model’s performance using appropriate metrics (e.g., accuracy, precision, recall, F1-score for classification; RMSE, MAE for regression).
• Hyperparameter Tuning
  • Optimize model performance by tuning hyperparameters through techniques like grid search, random search, or Bayesian optimization.
  • Validate the tuned model to ensure improved performance.
• Model Interpretation and Visualization
  • Interpret the AI model’s predictions and understand its decision-making process.
  • Visualize model outputs and insights to communicate findings effectively.
• Deployment and Integration
  • Prepare the AI model for deployment in a production environment.
  • Integrate the model into applications or systems through APIs or other deployment methods.
  • Ensure scalability, reliability, and efficiency of the deployed model.
• Testing and Validation
  • Conduct rigorous testing to validate the AI model’s functionality and performance in real-world scenarios.
  • Address any issues or bugs identified during testing phases.
• Documentation and Reporting
  • Document the entire AI project process, including data sources, preprocessing steps, model selection criteria, and evaluation results.
  • Prepare comprehensive reports or presentations summarizing the project outcomes, insights gained, and recommendations for stakeholders.
• Ethical Considerations
  • Consider ethical implications related to AI, such as fairness, transparency, privacy, and bias mitigation throughout the project lifecycle.
  • Implement measures to ensure responsible AI deployment and adherence to ethical guidelines.
• Iterative Improvement
  • Iterate on the AI model based on feedback, new data, or evolving business requirements to enhance its performance and relevance over time.

• Problem Identification and Definition
  • Define a clear problem statement or task that the AI project aims to address. This could involve tasks such as image classification, natural language processing, predictive analytics, etc.
  • Understand the scope, objectives, and constraints of the project.
• Data Collection and Preprocessing
  • Identify relevant data sources needed to train and evaluate the AI model.
  • Clean and preprocess the data to handle missing values, outliers, and inconsistencies.
  • Perform exploratory data analysis (EDA) to understand the characteristics and distribution of the data.
• Feature Engineering
  • Extract and select appropriate features from the data that are relevant to the problem statement.
  • Transform and engineer features to enhance the performance of the AI model.
• Model Selection
  • Choose suitable AI models based on the nature of the problem and data.
  • Experiment with different algorithms such as supervised (e.g., regression, classification), unsupervised (e.g., clustering), or reinforcement learning based on project requirements.
• Model Training and Evaluation
  • Split the dataset into training, validation, and test sets.
  • Train the selected AI model using the training data and validate its performance using the validation set.
  • Evaluate the model’s performance using appropriate metrics (e.g., accuracy, precision, recall, F1-score for classification; RMSE, MAE for regression).
• Hyperparameter Tuning
  • Optimize model performance by tuning hyperparameters through techniques like grid search, random search, or Bayesian optimization.
  • Validate the tuned model to ensure improved performance.
• Model Interpretation and Visualization
  • Interpret the AI model’s predictions and understand its decision-making process.
  • Visualize model outputs and insights to communicate findings effectively.
• Deployment and Integration
  • Prepare the AI model for deployment in a production environment.
  • Integrate the model into applications or systems through APIs or other deployment methods.
  • Ensure scalability, reliability, and efficiency of the deployed model.
• Testing and Validation
  • Conduct rigorous testing to validate the AI model’s functionality and performance in real-world scenarios.
  • Address any issues or bugs identified during testing phases.
• Documentation and Reporting
  • Document the entire AI project process, including data sources, preprocessing steps, model selection criteria, and evaluation results.
  • Prepare comprehensive reports or presentations summarizing the project outcomes, insights gained, and recommendations for stakeholders
• Ethical Considerations
  • Consider ethical implications related to AI, such as fairness, transparency, privacy, and bias mitigation throughout the project lifecycle.
  • Implement measures to ensure responsible AI deployment and adherence to ethical guidelines.
• Iterative Improvement
  • Iterate on the AI model based on feedback, new data, or evolving business requirements to enhance its performance and
• Problem Identification and Definition
  • Define a clear problem statement or task that the AI project aims to address. This could involve tasks such as image classification, natural language processing, predictive analytics, etc.
  • Understand the scope, objectives, and constraints of the project.
• Data Collection and Preprocessing
  • Identify relevant data sources needed to train and evaluate the AI model.
  • Clean and preprocess the data to handle missing values, outliers, and inconsistencies.
  • Perform exploratory data analysis (EDA) to understand the characteristics and distribution of the data.
• Feature Engineering
  • Extract and select appropriate features from the data that are relevant to the problem statement.
  • Transform and engineer features to enhance the performance of the AI model.
• Model Selection
  • Choose suitable AI models based on the nature of the problem and data.
  • Experiment with different algorithms such as supervised (e.g., regression, classification), unsupervised (e.g., clustering), or reinforcement learning based on project requirements.
• Model Training and Evaluation
  • Split the dataset into training, validation, and test sets.
  • Train the selected AI model using the training data and validate its performance using the validation set.
  • Evaluate the model’s performance using appropriate metrics (e.g., accuracy, precision, recall, F1-score for classification; RMSE, MAE for regression).
• Hyperparameter Tuning
  • Optimize model performance by tuning hyperparameters through techniques like grid search, random search, or Bayesian optimization.
  • Validate the tuned model to ensure improved performance.
• Model Interpretation and Visualization
  • Interpret the AI model’s predictions and understand its decision-making process.
  • Visualize model outputs and insights to communicate findings effectively.
• Deployment and Integration
  • Prepare the AI model for deployment in a production environment.
  • Integrate the model into applications or systems through APIs or other deployment methods.
  • Ensure scalability, reliability, and efficiency of the deployed model.
• Testing and Validation
  • Conduct rigorous testing to validate the AI model’s functionality and performance in real-world scenarios.
  • Address any issues or bugs identified during testing phases.
• Documentation and Reporting
  • Document the entire AI project process, including data sources, preprocessing steps, model selection criteria, and evaluation results.
  • Prepare comprehensive reports or presentations summarizing the project outcomes, insights gained, and recommendations for stakeholders.
• Ethical Considerations
  • Consider ethical implications related to AI, such as fairness, transparency, privacy, and bias mitigation throughout the project lifecycle.
  • Implement measures to ensure responsible AI deployment and adherence to ethical guidelines.
• Iterative Improvement
  • Iterate on the AI model based on feedback, new data, or evolving business requirements to enhance its performance and relevance over time.