Intelligence artificielle et données
488 documents à télécharger gratuitement
Cours, examens, TD, TP et exercices de intelligence artificielle et données. Thèmes couverts : machine learning, apprentissage automatique, big data, data science, fouille de données.
This document provides an in-depth critique of MapReduce's limitations, emphasizing its inefficiencies in complex, multi-step computations. It introduces Apache Spark as a more advanced, memory-optimized solution with higher performance, scalability, and interactive APIs for diverse programming languages. Key Spark components like RDDs, DataFrames, Datasets, and its ecosystem libraries (e.g., MLlib, GraphX) are detailed alongside their integration capabilities. The document further outlines Spark's architecture, including its execution model and resource management, while contrasting its pe...
This document discusses the limitations of MapReduce for complex tasks and introduces Apache Spark as a more efficient alternative. It covers the features, ecosystem, and architecture of Spark, emphasizing its ability to handle both batch and real-time processing. Spark is designed for speed, utilizing in-memory data storage, and supports various programming languages for ease of application development.
This paper investigates the properties of deep neural networks that contribute to their state-of-the-art performance on speech and visual recognition tasks. It highlights two counter-intuitive findings: the indistinguishability of individual high-level units and the effects of small perturbations on network predictions. The authors argue that it is the space of activations that retains semantic information rather than individual units.
This paper addresses the challenges associated with the explainability of AI and machine learning methods, particularly the issues arising from black box models. It contrasts the traditional view of explainable AI with a reality-based perspective, discussing the limitations and implications of explainable AI. The authors focus on the necessity and consequences of fostering transparency in AI decisions, particularly in critical fields such as health sciences and criminal justice.
This document explores the challenges and implications of providing explanations for automated decisions in the context of the GDPR. It discusses counterfactual explanations as a means to understand, contest, and alter decisions made by algorithms. The paper also highlights legal and technical barriers to implementing the right to explanation.
This paper discusses the challenges of evaluating defenses against adversarial examples in machine learning. The authors highlight common pitfalls in existing evaluations and propose a checklist to guide researchers in conducting more robust evaluations. The work aims to improve the understanding and completeness of defense evaluations among both developers and reviewers.
This paper discusses the vulnerability of neural networks to adversarial examples, which are inputs modified to mislead models. It asserts that this vulnerability stems from the linear nature of neural networks rather than previously suggested factors. The authors propose a method for generating adversarial examples that aids in enhancing model training.
The document explores various techniques for performing incremental ETL (Extract, Transform, Load) processes. It outlines three architectural approaches: single-stage ETL for small-scale BI solutions with minimal data sources, two-stage ETL for moderately complex or larger data volumes that a single stage cannot accommodate, and three-stage ETL for even more complex architectures that optimize data flow and reduce load on source systems. The content is designed to guide data engineers in building efficient ETL pipelines tailored to specific use cases.
The document explores incremental ETL techniques with three specific architectures: one-stage, two-stage, and three-stage ETL processes. A one-stage ETL is suitable for small-scale BI systems with limited data sources and simple requirements. Two-stage ETL addresses complexities and larger data volumes, optimizing performance. Lastly, a three-stage ETL architecture reduces extraction overhead and minimizes source system load, enhancing efficiency in data flow management.
This document provides an overview of Big Data, including its definition, sources, challenges, and the 5 Vs that characterize it. It highlights the importance of data in decision-making processes and the necessity of effectively managing and analyzing this data. Additionally, it discusses the rapid generation of large volumes of unstructured data and the implications for businesses and technology.
This article discusses the limitations of deep learning in AI, particularly its lack of interpretability and ability to answer logical questions. It introduces the concept of Neurosymbolic AI, which combines aspects of symbolic AI with deep learning to provide more explainable and interpretable AI systems. The text highlights the historical context of AI development and contrasts traditional symbolic AI with modern deep learning approaches.
The document outlines a two-part exam focused on designing and implementing a data warehouse and building an ETL process. The first part involves creating a new data warehouse schema, defining a time dimension table using Transact-SQL scripts, and populating it with dates for a specific year. The second part requires developing an ETL solution by restoring a database, analyzing its content to generate sample data and statistical reports, and performing data profiling tasks on key database tables. Key deliverables include multiple SQL scripts, output files, and profiling reports stored in sp...
The document provides a detailed hands-on exercise to optimize SSIS packages for incremental data extraction, employing methods like using a datetime column, Change Data Capture (CDC), CDC Control Tasks, and Change Tracking for efficient ETL operations. The first exercise demonstrates the use of datetime columns to extract modified records and implement the high water mark technique. The second exercise uses SQL Server's CDC feature to retrieve only modified rows from a database without utilizing a dedicated timestamp column, while also including a workflow for testing changes. Together, th...
This document outlines a lab exercise for optimizing SQL Server Integration Services (SSIS) packages to improve ETL processes by enabling incremental data extraction. The lab focuses on using datetime columns, Change Data Capture (CDC), and Change Tracking mechanisms to filter out previously extracted data, reducing system load and resource consumption. The steps include configuring SSIS controls, SQL queries, and variables to implement the 'high-water mark' technique. Additionally, the lab demonstrates enabling SQL Server's CDC feature and creating a stored procedure to retrieve modified r...
This document provides a comprehensive guide to implementing incremental ETL processes, focusing on techniques for identifying and extracting modified or inserted records. It explores multiple strategies including the use of DateTime columns, Change Data Capture (CDC), and Change Tracking features of SQL Server. Detailed methods for configuration, data extraction, and error handling are outlined, especially when leveraging SQL Server's CDC and SSIS functionalities. Finally, it discusses techniques to handle deleted records and optimize processing efficiency during data warehouse refresh cyc...
This document focuses on implementing incremental ETL processes to extract modified data from source systems. Techniques such as using primary keys with checksums, datetime columns, Change Data Capture (CDC), and Change Tracking in SQL Server are discussed in detail. It includes step-by-step processes for configuration and extraction using CDC and Change Tracking. Additionally, it provides guidance on methods to identify record changes, whether inserted, updated, or deleted, and how to handle logical and physical deletions efficiently.
This document explores the process of implementing a data extraction solution, specifically focusing on incremental ETL strategies for refreshing data warehouses. It details the challenges and best practices for tracking data modifications, ensuring dimensional consistency, and maintaining historical integrity. The text further breaks down common ETL architectures into single-stage, two-stage, and three-stage processes and highlights considerations for planning data extraction windows and transformations. Finally, it addresses strategies on managing dependencies, surrogate keys, and optimiz...
The document discusses designing an effective incremental ETL process for refreshing data warehouses with new or modified data from source systems. It outlines key considerations for planning such processes, including managing data modifications, understanding dimension keys, and handling updates to dimension and fact records. The document also details common ETL data flow architectures—single-stage, two-stage, and three-stage—and provides guidelines for planning data extraction windows and transformations to enhance performance and data consistency.
This document outlines key considerations and methodologies for designing and implementing an incremental ETL process to refresh a data warehouse, focusing on efficient data extraction, transformation, and load cycles. It elaborates on single-, two-, and three-stage ETL architectures, each intended for scaling throughput and maintaining data integrity across systems. The document also highlights the need to handle data modifications (e.g., inserts, updates, deletions), plan refresh schedules, and manage dependencies between fact and dimension data for accurate reporting. Guidelines for dete...
This module focuses on planning an incremental ETL process for data extraction in data warehousing. It covers the key considerations and architectures involved in effective data extraction strategies. By the end of the module, learners will understand how to implement a robust data warehouse refresh solution that maximizes performance.
This document discusses the planning and implementation of data extraction solutions within data warehousing contexts. It emphasizes the importance of incremental ETL processes, outlining their challenges and considerations for effective data refresh cycles. The document concludes with key objectives and considerations necessary for a successful incremental ETL implementation.
This module provides an overview of implementing a data extraction solution in data warehousing. It focuses on the incremental ETL process and its challenges, providing guidance on effective planning and architecture. The lesson aims to equip learners with the knowledge to design efficient data warehouse refresh solutions.
This paper introduces the CLEVRER dataset designed to evaluate computational models on temporal and causal reasoning tasks in videos. It emphasizes the importance of understanding the underlying causal structures in video data, contrasting with the predominant focus on visual pattern recognition. The study identifies the limitations of current models in handling causal reasoning tasks, proposing a new benchmark for future research.
This document explores the application of regular expressions in R for processing email data. Initially, challenges in parsing email domains are identified, leading to the utilization of specific regular expressions to extract data systematically. Functions like 'grep', 'gsub', 'regexpr', and 'regmatches' are discussed and applied to manipulate, search, and modify email strings effectively. Results include generating a frequency table for email domains from the processed dataset.





















