Introduction to Machine Learning

Machine learning must be one of the fastest growing fields in computer science. It is not only that the data is continuously getting “bigger,” but also the theory to process it and turn it into knowledge.

In various fields of science, from astronomy to biology, but also in everyday life, as digital technology increasingly infiltrates our daily existence, as our digital footprint deepens, more data is continuously generated and collected.

Whether scientific or personal, data that just lies dormant passively is not of any use, and smart people have been finding ever new ways to make use of that data and turn it into a useful product or service. In this transformation, machine learning plays a larger and larger role.

Every year, datasets are getting larger. Not only has the number of observations grown, but the number of observed attributes has also increased significantly. There is more structure to the data: It is not just numbers and character strings any more but im- ages, video, audio, documents, web pages, click logs, graphs, and so on.

More and more, the data moves away from the parametric assumptions we used to make-for example, normality. Frequently, the data is dynamic and so there is a time dimension. Sometimes, our observations are multi-view-for the same object or event, we have multiple sources of information from different sensors and modalities.

Our belief is that behind all this seemingly complex and voluminous data, there lies a simple explanation. That although the data is big, it can be explained in terms of a relatively simple model with a small number of hidden factors and their interaction. Think about millions of customers who each day buy thousands of products online or from their local super- market. This implies a very large database of transactions, but there is a wealth of valuable information hidden within this data that can be leveraged using machine learning (ML) techniques.

Extracting Insights from Transaction Data

Every purchase, whether online or in-store, generates data points that can be analyzed to uncover patterns, trends, and customer preferences. Machine learning algorithms can process this vast amount of transactional data to:

1. Personalized Recommendations

   • Systems like those used by Amazon and Netflix analyze past purchases or viewing habits to suggest relevant products or content.

   • Collaborative filtering and matrix factorization techniques help predict what a customer might like based on similar users’ behavior.

2. Market Basket Analysis

   • By applying association rule learning (e.g., Apriori algorithm), retailers can identify frequently co-purchased items (e.g., “Customers who bought bread also bought butter”).

   • This helps in optimizing product placements, promotions, and bundling strategies.

3. Customer Segmentation

   • Clustering algorithms (like K-means or DBSCAN) group customers based on purchasing behavior, demographics, or spending habits.

   • Businesses can then tailor marketing campaigns for different segments (e.g., high-value customers, occasional shoppers).

4. Demand Forecasting

   • Time-series forecasting models (such as ARIMA or LSTM neural networks) predict future sales trends, helping retailers manage inventory efficiently.

   • This reduces stockouts and overstocking, optimizing supply chain operations.

5. Fraud Detection

   • Anomaly detection algorithms (like Isolation Forest or Autoencoders) flag suspicious transactions in real time, preventing financial losses.

   • ML models learn from historical fraud patterns to improve accuracy over time.

Challenges in Handling Transaction Data

Despite its potential, working with large-scale transaction data presents challenges:

• Data Volume & Scalability – Processing millions of transactions requires distributed computing frameworks like Apache Spark.

• Data Sparsity – Not all customers buy all products, leading to sparse matrices that need specialized ML techniques.

• Real-Time Processing – For instant recommendations or fraud detection, models must process data in milliseconds (e.g., using streaming analytics).

The Future of ML in Retail & E-Commerce

With advancements in deep learning, reinforcement learning, and generative AI, the future holds even more possibilities:

• Dynamic Pricing – AI adjusts prices in real time based on demand, competition, and customer behavior.

• Visual Search & AI-Powered Shopping Assistants – Customers can upload images to find similar products or get personalized styling advice.

• Cashierless Stores – Computer vision and sensor fusion enable seamless checkout experiences (e.g., Amazon Go).

By harnessing machine learning, businesses can transform raw transaction data into actionable insights, enhancing customer experiences while driving efficiency and profitability.

Introduction to Machine Unlearning

Recently, there has been an increasing emphasis on laws governing user privacy. For instance, the General Data Protection Regulation (GDPR), known as the right to be forgotten, mandates that machine learning applications must remove specific data upon a user’s request and retrain their models accordingly.

Additionally, from a security standpoint, it is crucial to protect training data, which may contain sensitive user information, including ensuring its proper erasure. Once users provide their information to an application, it becomes challenging to completely eliminate it.

In the current landscape where machine learning is prevalent, many advanced features rely heavily on the data they process and learn from. Consequently, users’ privacy is deeply embedded within these applications, making it more susceptible to unauthorized access and data breaches.

EWC Loss

Catastrophic unlearning is a challenge in developing deep learning models capable of continuous learning. While learning a new task, the model may gradually forget knowledge from previous tasks. This means that prior knowledge may diminish over time.

Recent studies have shown that continuous learning is easy when tasks have high similarity, and becomes difficult when similarity has diminished.

Ramasesh et al. presented an empirical study of catastrophic forgetting in this pre-training paradigm, finding that in the pre-trained model, the different class representations become more orthogonal with increasing size. We incorporate EWC Loss into the loss computation of machine unlearning to avoid “catastrophic unlearning”.

EWC preserves important parameter information of the previous task and helps to maintain the performance of the previous task when learning a new task.

This approach allows the model to learn incrementally without re-learning all tasks. In terms of algorithm, EWC only needs to compute and store the diagonal elements of the Fisher information matrix of the parameters, instead of storing the complete Fisher information matrix. It makes more efficient in terms of memory and computational efficiency.

Difference Between Machine Learning and Machine Unlearning

Here are the key differences between machine learning and machine unlearning

Aspect	Machine Learning	Machine Unlearning
Purpose	Learn patterns from data to make predictions.	Remove specific data’s influence from a model.
Data Focus	Adding and using data to train models.	Selectively removing or “forgetting” data.
Motivation	Improve model performance and generalization.	Privacy, compliance, or bias correction.
Complexity	Focuses on optimization and accuracy.	Requires balancing data removal with model utility.
Example Use Case	Training a chatbot to respond to user queries.	Deleting a user’s chat history from the model.
Challenges	Generalization, overfitting	Efficient deletion, maintaining model integrity
Methods	Supervised, Unsupervised, Reinforcement Learning	Exact retraining, Approximate forgetting

 

Challenges in Machine Unlearning

• Efficiency: Completely retraining a model without the target data is computationally expensive, so unlearning techniques aim to approximate this efficiently.
• Accuracy Trade-off: Removing data may degrade model performance if the data was significant to the model’s learning.
• Verification: Ensuring the data has been effectively “forgotten” is difficult, as traces may remain in the model’s parameters.

In summary, machine learning is about building models by learning from data, while machine unlearning is about selectively erasing the influence of specific data from those models, often for ethical or legal reasons.