Schedules

Language models based on deep learning are widely used in a number of applications, including text summarization, conversational bots, Q&A, text generation, translation, semantic search, information retrieval etc. Many research and industry participants use pre-trained language models (PLMs) to build and architect these use cases. With the widespread use of pre-trained language models (PLM), there has been increased research on how to make them applicable, especially in limited resource or low latency high throughput scenarios. One of the dominant approaches is knowledge distillation (KD), where a smaller model is trained by receiving guidance from a large PLM. While there are many successful designs for learning knowledge from teachers, it remains unclear how students can learn better. Inspired by real university teaching processes, in this work we further explore knowledge distillation and propose a very simple yet effective framework, SEAD, to further improve task-specific generalization by utilizing multiple teachers. Our experiments show that SEAD leads to better performance compared to other popular KD methods and achieves comparable or superior performance to its teacher model such as BERT on total 13 tasks for the GLUE and SuperGLUE benchmarks.

As we embark on an era of data-driven decisions, preserving the privacy of the underlying data is of utmost importance. Differential privacy has emerged as the go-to solution for tech giants and data vendors as it not only provides mathematical definition to privacy but also grants the ability to control the privacy parameter (noise) in the underlying data. The differentially private data aggregates sensitive individual data over multiple features and masks them with statistical noise. The conventional ML algorithms need to be tweaked to handle such aggregated data with noise. In this paper, probabilistic and Isolated learning techniques are leveraged to model differentially private data to improve the click-through rate of Ad-tech campaigns. The ability to predict the click event for different feature combinations such as URL, Ad type, Device type, etc. beforehand from the aggregated noisy data will help in developing a custom bidding strategy in a highly competitive programmatic setting (Real-time bidding)

Forecasting plays a significant role in making effective business decisions. Many products are launched in the market every year and the initial sales of the products are very much essential for gaining insights on the sustainability of the product in the market. But because of the data limitation, forecasting with traditional models becomes a huge problem and it is fraught with risks and hence, estimates can often be off the mark. So, generating accurate forecasts becomes crucial as they can help the companies in assessing overall performance, budgeting, risk management, and cost reduction. In this paper, Forecasting of Recently Launched Products with limited data using different techniques is presented. Multiple approaches have been tested and compared to a baseline strategy to solve the above problem. For testing, the results from all the models implemented have been compared, using various statistical error measures such as the Mean Absolute Percentage Error (MAPE), the Weighted Mean Absolute Percentage Error (WAPE), and Mean Absolute error (MAE).

In this paper, we present our approach on using time series forecasting for managing resource allocation for one of the Social Media UPI. With the rapid increase in online payments, the business is facing challenges in resource allocation across three vendors to monitor its payment requests categorized under seven hierarchical processes labelled as Towers. Each of these Towers is further divided into various sub-processes called Products and then Alert Queues. Since the workforce allocation depends upon the incoming volume requests for these processes, it is vital to have a proper volume forecasting system to plan for the optimal resource allocation in advance. We forecasted volumes for all Products and Alert Queues across all the Towers and categorized them under three different risk buckets depending upon the Mean Absolute Percentage Error (MAPE) observed on test data. Using the forecasted volumes and the risk category the Product falls under, capacity planning for the future dates is made easier and the client was able to reduce its resources by 26%.

Deep Learning has revolutionized the fields of computer vision, natural language understanding, speech recognition, information retrieval and more. Many techniques have evolved over the past decade that made models lighter, faster, and robust with better generalization. However, many deep learning practitioners persist with pre-trained models and architectures trained mostly on standard datasets such as Imagenet, MS-COCO, IMDB-Wiki Dataset, and Kinetics-700 and are either hesitant or unaware of redesigning the architecture from scratch that will lead to better performance. This scenario leads to inefficient models that are not suitable on various devices such as mobile, edge, and fog. In addition, these conventional training methods are of concern as they consume a lot of computing power. In this paper, we revisit various SOTA techniques that deal with architecture efficiency (Global Average Pooling, depth-wise convolutions & squeeze and excitation, Blurpool), learning rate (Cyclical Learning Rate), data augmentation (Mixup, Cutout), label manipulation (label smoothing), weight space manipulation (stochastic weight averaging), and optimizer (sharpness aware minimization). We demonstrate how an efficient deep convolution network can be built in a phased manner by sequentially reducing the number of training parameters and using the techniques mentioned above. We achieved a SOTA accuracy of 99.2% on MNIST data with just 1500 parameters and an accuracy of 86.01% with just over 140K parameters on the CIFAR-10 dataset.

Document Similarity could be a building block for many useful applications, Information Retrieval, Document Clustering, Question-Answering Systems to name a few. In modern digital world, Informative Documents are composed of Text, Images and Videos. In such a scenario, similarity based on purely Text, Image or Video may not be adequate. Hence a metrics blending similarity on all these aspects should be used. In this paper, a weighted similarity measure based on Texts and Images has been developed, using some popular opensource Machine Learning (ML) libraries. This provides a flexible and easy method, without using large training data, which often is the case with ML tasks.

Pharmacovigilance has gained much prominence to identify the adverse signal originating from the use of a drug and subsequent reviews. Pharmacovigilance analytics can be applied to gain insights by integrating data related to medical products from multiple sources and applying techniques to search, compare and summarize them. Identifying the drug adversities in prior with the help of pharmacovigilance analytics can help to reduce the risk of unfavorable outcomes in selecting line of treatment. However multiple sources (medical reports, review websites, online platforms, etc.) exist for adverse signal detection which makes it challenging to extract the information of interest manually. The gap is readily identified by the authors and is addressed in this work by proposing a temporal graph neural network based approach. The existing Named Entity Recognition (NER) techniques are not efficient enough to Identify and extract the causal relationship between the drug and the events based on but not limited to onset of the event, giving insights like type of ADR. In contrast, we propose a temporal graph neural network-based approach which includes both entity recognition as well as event extraction followed by finding the causal relationship between the drug and the events. The technique uses temporal topological information, word dependency parsing, edge information, which is representing the relationship among the entities, Node Embeddings which are vectors which reflect properties of nodes in a network. The proposed methodology has exhibited improved result in comparison to the state of the art and has shown statistical significance in performance measure.

The credit card fraud detection at transaction level is a fairly complex problem in the consumer lending industry across the globe. This classification problem combines the intricacies of finding a needle in a haystack (highly imbalanced classes) and hitting a moving target (dynamic fraud pattern recognition) at the same time. To add to the issues, any wrong decision (misclassification) in approving or declining a credit card transaction either leads to an immediate and direct monetary loss to the lender or compromises the core of the business – “the genuine customer’s experience”. The credit card industry has come a long way in identifying fraudulent transactions by adopting various advanced statistical techniques over years. The fraudsters, however, are not laggards. They invent channels and target new loopholes. The alerts generated by analysis of location and amount of a transaction do not suffice anymore. This paper aims at a) highlighting the significance of feature engineering in credit card fraud pattern recognition based on our industry experience, corroborated by deeper analysis of the information collected for credit card transactions, and b) comparing performance of different machine learning algorithms and quantifying their value addition to naïve models.

Millions of packages are delivered successfully by online and local retail stores across the world every day. The proper delivery of packages is needed to ensure high customer satisfaction and repeat purchases. These deliveries suffer various problems despite the best efforts from the stores. These issues happen not only due to the large volume and high demand for low turnaround time but also due to mechanical operations and natural factors. These issues range from receiving wrong items in the package to delayed shipment to damaged packages because of mishandling during transportation. Finding solutions to various delivery issues faced by both sending and receiving parties plays a vital role in increasing the efficiency of the entire process. This paper shows how to find these issues using customer feedback from the text comments and uploaded images. We used transfer learning for both Text and Image models to minimize the demand for thousands of labeled examples. The results show that the model can find different issues. Furthermore, it can also be used for tasks like bottleneck identification, process improvement, automating refunds, etc. Compared with the existing process, the ensemble of text and image models proposed in this paper ensures the identification of several types of delivery issues, which is more suitable for the real-life scenarios of delivery of items in retail businesses. This method can supply a new idea of issue detection for the delivery of packages in similar industries.

Incidents leading to a health and safety risk are a common occurrence in most large-scale Construction & Engineering work sites, which can lead to millions of dollars in damages and lawsuits affecting financial performance of the overall project. Early detection of these potential issues and proactive interventions could lead to avoiding of accidents or safety breaches that can occur in worksite, which will not only have huge financial benefits but also will ensure timely delivery of projects. We, part of the Construction Engineering Global Business Unit (CEGBU) in Oracle, have applied NLP based state-of-the-art Machine Learning (ML) models to classify text data from textual construction injury reports as well as correspondence data between construction project participants. The health and safety risk detection sub-system can predict if the text data is associated with (any impending) risks with high accuracy. The sub-system also enables human decision- maker to provide feedback for these correspondences, based on human experience and intuition, in case the prediction(s) made by the subsystem is incorrect. This feedback will later be fed back to the sub–system as (new) labelled training data to improve the prediction accuracy and re-inforce the sub-system over time.

Interpreting image data to neural networks is challenging. Deep convolutional neural network methods have shown promising results in interpreting image data to neural networks, with convolution and pooling operations over the traditional fully connected dense layers. The performance of these deep convolutional methods, however, is often compromised by the constraint that the convolution and pooling operations interpret the image data to neural networks by compressing the data into lower dimensions that lead to information bottleneck. To mitigate this, neural networks tend to be larger with more parameters (filters) thus increasing the computational cost (GPU Resources). In this paper, I present an information-preserving way to interpret image data to a convolutional neural net without any information bottleneck and with relatively fewer parameters, that also ensures no loss in information due to convolution or pooling operations. Uniquely my method adds two additional operations, one over the first regular convolution operation and the other operation next to the deeper convolution operation in the neural net. The first operation is to divide the image into individual frames by frame-based crop operation and then apply regular convolution, pooling operations to interpret individual frames of the image into low dimensional tensors that preserve information from being bottlenecked since operations use frames of the image instead of using total image data. The second operation is a convolution operation applied after joining all low dimensional tenors of individual frames to extract information that later passes through the rest of the layers. This idea of using frames can better help model tasks like image generation and completion as well. Using frames of data instead of the total image helps in parallelizing computations thereby drastically decreasing the computational cost and depth of a neural network. Experiments conducted on inception networks worked better with relatively small network architecture on vision tasks.

This problem concerns with predicting whether the patient will get stroke in the future with predictors like age, gender, smoking status, body mass index, whether they had heart disease, whether they had hypertension etc. Since the output variable is categorical in nature, it is a classification problem. Many classification techniques are used with the help of three main Business Analytics tools such as Excel, R and Python. The data is understood through Exploratory Data Analysis, then Data Pre-Processing is done to prepare the data, various models are built on the data and finally Error metrics are used to compare the results.

Despite the advances in data acquisition, storage and algorithm capabilities, the impact of Machine Learning (ML) initiatives is sometimes debated across sponsoring units and organizations. While there is a possibility of the data and models not having the desired predictive value, there is a possibility that the desired business value is not realized due to an improper implementation of the interventions from the predictions intended. We discuss the chronological nature of data richness and the increase in predictive capability, and conversely there exists an inverse relationship between most effective interventions and the richness of data available for actionable predictions across process & product lifecycles. It thus becomes imperative to view ML implementations not as technology implementations but as Business Process Re- engineering initiatives to deliver the optimal business / process returns.