Intelligence artificielle et données
488 documents à télécharger gratuitement
Cours, examens, TD, TP et exercices de intelligence artificielle et données. Thèmes couverts : machine learning, apprentissage automatique, big data, data science, fouille de données.
This document provides an overview of decision trees with a specific focus on the Gini Index as a metric. A banking use case illustrates predicting customers' loan repayment capacity through a decision tree framework. The step-by-step construction of the decision tree is documented, emphasizing rules extraction. Strengths and weaknesses of decision trees are critically analyzed, shedding light on their advantages and inherent limitations.
This document discusses decision trees and the Gini index in the context of predicting the ability of clients to repay loans. It provides examples and steps for constructing decision trees. Strengths and weaknesses of decision trees are also outlined, highlighting their interpretability and challenges with small datasets.
This document thoroughly discusses the interdisciplinary field of data mining, blending statistics and information technology, focusing on uncovering hidden patterns in large datasets. It elaborates on various applications, such as financial fraud detection and medical diagnostics, showcasing its impact across industries. Methodologies like KDD, SEMMA, and CRISP-DM structure data mining workflows, while techniques include predictive (supervised learning) and descriptive (unsupervised learning) approaches. Practical examples provide insights into decision-assistive processes powered by algor...
This document provides an introduction to data mining, focusing on its definition, interdisciplinary nature, and applications across various domains. It highlights the methods and algorithms involved in extracting meaningful insights from vast datasets and emphasizes the importance of data preparation. The applications of data mining include fraud detection, risk management, customer analysis, and healthcare diagnostics.
The document outlines two exercises focusing on predictive modeling and statistical analysis. Exercise 1 requires constructing a decision tree using CART or Entropy for predicting company profitability and evaluating its performance. Exercise 2 involves applying simple linear regression to analyze the relationship between capital stock and company investment based on data from 27 industrial companies. The results from a data science software provide statistical estimates to assess regression quality and interpret findings.
This document outlines an exam focusing on decision trees and linear regression. It includes exercises on decision tree algorithms and the application of linear regression using real data. Students are tasked with evaluating methods and interpreting statistical results.
This document provides a comprehensive overview of classification and clustering methods in data science. It distinguishes between supervised and unsupervised learning, explaining criteria for optimal clustering with similarity measures and specific metrics like Manhattan and Euclidean distances. The non-hierarchical methods, including k-means clustering and dynamic clouds, emphasize minimizing intra-class inertia and provide examples of their application in market segmentation. Hierarchical clustering techniques are also explored, including dendrogram construction to visualize classificati...
This document is a practical exercise focusing on Correspondence Analysis (AFC), likely targeting master’s level students at the Université Virtuelle de Tunis in a BADS program. It may cover the methodology of AFC, including how to perform the analysis using real-world datasets and software tools. Through guided examples and exercises, the material teaches students how to interpret the results and draw insightful conclusions. The findings aim to equip students with practical knowledge essential for quantitative analysis in various fields.
This document introduces and applies correspondence analysis (CA) to multidimensional contingency tables. The methodology involves partitioning data into classes, visualizing their positions on principal component graphs, and exploring hierarchical relationships through ultrametric distances. Findings include detailed correspondence patterns and profiles derived from respondent data, along with specific metrics such as eigenvalues and chi-squared statistics, which provide insights into variable relationships.
This document covers concepts related to multidimensional analysis, including class partitioning and ultrametric distance evaluation. It presents an exercise involving brand name selection based on customer preferences using multidimensional data analysis methods. The results are summarized in tables and graphical representations, highlighting frequency distributions.
This document presents a multidimensional analysis of food data and socio-professional categories. It includes correlation matrices and results from Principal Component Analysis (PCA). The aim is to study the relationships between different food categories and socio-professional groups.
This document discusses the concept of adversarial examples in machine learning, detailing their implications for model security and performance. It explores the historical context of adversarial attacks, various methods for crafting such examples, and the defensive strategies available. Additionally, it highlights the potential of adversarial examples to improve machine learning models even in the absence of an adversarial actor.
This research presents a novel detection method using Shapley Additive Explanations (SHAP) values to distinguish between normal and adversarial inputs in deep neural networks (DNNs). The method is evaluated using a dataset of adversarial examples created from CIFAR-10 and MNIST datasets, demonstrating high detection accuracy and generalization across diverse attack methods. The study highlights the connection between model explainability and robustness against adversarial perturbations.
This document reviews the recent advances in Explainable Artificial Intelligence (XAI), focusing on the interpretability methods used in machine learning models. It highlights the challenge of model complexity leading to 'black box' systems, causing difficulties in understanding their decision-making processes. The survey aims to provide a comprehensive taxonomy and programming implementation links to serve as a reference for researchers and practitioners in the field.
Hive is a tool designed for distributed SQL querying over a Hadoop-based cluster, translating SQL queries into MapReduce or Tez jobs. It uses HiveQL, an SQL-like language optimized for OLAP and large-scale data warehousing, supporting extensibility through user-defined functions and JDBC/ODBC drivers. While Hive simplifies OLAP queries and offers partitioning for data optimization, it has limitations such as high latency, lack of transactions, and capped SQL functionality compared to alternatives like Pig or Spark. Hive enables structured data management over unstructured data stored in Had...
Apache Pig is a high-level platform for creating MapReduce programs used with Hadoop, utilizing a procedural language called Pig Latin. Pig Latin abstracts Java-based MapReduce programming into a more flexible and SQL-like syntax with control over execution flow and data transformations. It supports parallel processing, user-defined functions (UDFs) in multiple languages, and handles large-scale datasets efficiently while combining aspects of both declarative SQL and procedural MapReduce. Advantages include a quick learning curve, flexibility in data processing, and the ability to embed cus...
This document introduces the MapReduce paradigm, a distributed computing model to process large datasets, especially in scenarios involving tasks such as data aggregation and analysis. Traditional sequential methods are contrasted with the efficiency and parallelization enabled by MapReduce, with emphasis on how large-scale problems can be divided into smaller subtasks. The two core operations, MAP and REDUCE, are described in depth, along with examples in word frequency counting, web analytics, and finding mutual connections in social graphs. The advantage of automating data distribution a...
This document explores the MapReduce paradigm, designed for processing large data sets through distributed computing. It discusses the traditional sales aggregation problem, outlining inefficiencies and presenting MapReduce as a solution. The framework allows for efficient problem-solving by breaking down tasks and parallelizing the computation necessary for big data analysis.
This document provides an in-depth explanation of the MapReduce paradigm used in large-scale distributed computing. It describes the inefficiencies of traditional solutions to analyze large datasets and introduces MapReduce as an effective alternative. The methodology involves dividing tasks into smaller, parallelizable operations: `MAP` to transform data into key-value pairs and `REDUCE` to aggregate results by key. Various examples, including word frequency analysis, website statistics, and common connections in social graphs, illustrate practical applications of the MapReduce model. The...
This document provides an overview of the MapReduce programming model and its efficiency in processing large datasets. It discusses the traditional methods of data processing and introduces the MapReduce paradigm, emphasizing its divide and conquer approach. The document outlines the operations of MAP and REDUCE, detailing the steps involved in the MapReduce process.
Hadoop is an open-source framework designed for distributed data processing across massive datasets, with scalable and fault-tolerant architecture. Core components include HDFS for distributed storage and MapReduce for distributed computing, with additional tools like Hive, Pig, and HBase for advanced analytics and data management. Inspired by Google's publications, Hadoop leverages principles like data redundancy, clustering, and parallel processing to support Big Data challenges. It has been widely adopted across industries, including by organizations such as Facebook, Google, and Amazon.
This document provides a detailed overview of Hadoop, an open-source framework designed for processing large volumes of data. It discusses its architecture, history, and key features, including performance, fault tolerance, and scalability. Additionally, it highlights major companies utilizing Hadoop and its ecosystem.
Hadoop, an open-source framework managed by Apache and written in Java, facilitates distributed processing of massive datasets using clusters of commodity hardware. Key components include HDFS for distributed storage and MapReduce for parallelized data processing, ensuring fault tolerance, scalability, and cost-efficiency. The ecosystem extends functionality through tools such as Pig, Hive, and Flume for data processing, storage, scheduling, and monitoring. Developed by Doug Cutting and inspired by Google technologies, Hadoop powers numerous enterprises like Amazon, Adobe, Facebook, and Twi...
Hadoop is an open-source framework designed for distributed processing of large data sets across clusters. It facilitates the creation of applications that handle massive volumes of data while ensuring performance and fault tolerance. The technology has evolved under the Apache foundation and is widely used in various enterprises.