b'Ali Ghodsi'

research

∙ 09/16/2023

Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference Using Sorted Fine-Tuning (SoFT)

The rapid advancement of large language models (LLMs) has revolutionized...

0 Parsa Kavehzadeh, et al. ∙

research

∙ 09/01/2023

SortedNet, a Place for Every Network and Every Network in its Place: Towards a Generalized Solution for Training Many-in-One Neural Networks

As the size of deep learning models continues to grow, finding optimal m...

0 Mojtaba Valipour, et al. ∙

research

∙ 04/22/2023

Recurrent Neural Networks and Long Short-Term Memory Networks: Tutorial and Survey

This is a tutorial paper on Recurrent Neural Network (RNN), Long Short-T...

0 Benyamin Ghojogh, et al. ∙

research

∙ 01/27/2023

Improved knowledge distillation by utilizing backward pass knowledge in neural networks

Knowledge distillation (KD) is one of the prominent techniques for model...

0 Aref Jafari, et al. ∙

research

∙ 12/12/2022

Continuation KD: Improved Knowledge Distillation through the Lens of Continuation Optimization

Knowledge Distillation (KD) has been extensively used for natural langua...

0 Aref Jafari, et al. ∙

research

∙ 12/12/2022

Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging

Knowledge Distillation (KD) is a commonly used technique for improving t...

0 Peng Lu, et al. ∙

research

∙ 10/14/2022

DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation

With the ever-growing size of pre-trained models (PMs), fine-tuning them...

0 Mojtaba Valipour, et al. ∙

research

∙ 05/25/2022

Towards Understanding Label Regularization for Fine-tuning Pre-trained Language Models

Knowledge Distillation (KD) is a prominent neural model compression tech...

0 Ivan Kobyzev, et al. ∙

research

∙ 05/14/2022

The Sky Above The Clouds

Technology ecosystems often undergo significant transformations as they ...

0 Sarah Chasins, et al. ∙

research

∙ 03/17/2022

When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation

Data Augmentation (DA) is known to improve the generalizability of deep ...

0 Ehsan Kamalloo, et al. ∙

research

∙ 10/16/2021

Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher

With ever growing scale of neural models, knowledge distillation (KD) at...

4 Mehdi Rezagholizadeh, et al. ∙

research

∙ 09/21/2021

Knowledge Distillation with Noisy Labels for Natural Language Understanding

Knowledge Distillation (KD) is extensively used to compress and deploy l...

0 Shivendra Bhardwaj, et al. ∙

research

∙ 09/13/2021

KroneckerBERT: Learning Kronecker Decomposition for Pre-trained Language Models via Knowledge Distillation

The development of over-parameterized pre-trained language models has ma...

0 Marzieh S. Tahaei, et al. ∙

research

∙ 09/13/2021

How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding

Knowledge Distillation (KD) is a model compression algorithm that helps ...

5 Tianda Li, et al. ∙

research

∙ 06/27/2021

Legendre Deep Neural Network (LDNN) and its application for approximation of nonlinear Volterra Fredholm Hammerstein integral equations

Various phenomena in biology, physics, and engineering are modeled by di...

0 Zeinab Hajimohammadi, et al. ∙

research

∙ 06/27/2021

SymbolicGPT: A Generative Transformer Model for Symbolic Regression

Symbolic regression is the task of identifying a mathematical expression...

0 Mojtaba Valipour, et al. ∙

research

∙ 05/28/2021

Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax

In Natural Language Processing (NLP), finding data augmentation techniqu...

0 Ehsan Kamalloo, et al. ∙

research

∙ 04/14/2021

Annealing Knowledge Distillation

Significant memory and computational requirements of large deep neural n...

9 Aref Jafari, et al. ∙

research

∙ 11/12/2020

Symbolically Solving Partial Differential Equations using Deep Learning

We describe a neural-based method for generating exact or approximate so...

0 Maysum Panju, et al. ∙

research

∙ 11/04/2020

A Neuro-Symbolic Method for Solving Differential and Functional Equations

When neural networks are used to solve differential equations, they usua...

0 Maysum Panju, et al. ∙

research

∙ 09/22/2020

Stochastic Neighbor Embedding with Gaussian and Student-t Distributions: Tutorial and Survey

Stochastic Neighbor Embedding (SNE) is a manifold learning and dimension...

0 Benyamin Ghojogh, et al. ∙

research

∙ 09/17/2020

Multidimensional Scaling, Sammon Mapping, and Isomap: Tutorial and Survey

Multidimensional Scaling (MDS) is one of the first fundamental manifold ...

0 Benyamin Ghojogh, et al. ∙

research

∙ 06/30/2020

Segmentation Approach for Coreference Resolution Task

In coreference resolution, it is important to consider all members of a ...

0 Aref Jafari, et al. ∙

research

∙ 04/17/2019

DeepNovoV2: Better de novo peptide sequencing with deep learning

We introduce DeepNovoV2, the state-of-the-art neural networks based mode...

0 Rui Qiao, et al. ∙

research

∙ 12/18/2018

Deep Variational Sufficient Dimensionality Reduction

We consider the problem of sufficient dimensionality reduction (SDR), wh...

0 Ershad Banijamali, et al. ∙

research

∙ 11/07/2018

SRP: Efficient class-aware embedding learning for large-scale data via supervised random projections

Supervised dimensionality reduction strategies have been of great intere...

0 Amir-Hossein Karimi, et al. ∙

research

∙ 07/23/2018

Text Classification based on Multiple Block Convolutional Highways

In the Text Classification areas of Sentiment Analysis, Subjectivity/Obj...

0 Seyed Mahdi Rezaeinia, et al. ∙

research

∙ 12/15/2017

A Berkeley View of Systems Challenges for AI

With the increasing commoditization of computer vision, speech recogniti...

0 Ion Stoica, et al. ∙

research

∙ 11/24/2017

JADE: Joint Autoencoders for Dis-Entanglement

The problem of feature disentanglement has been explored in the literatu...

0 Ershad Banijamali, et al. ∙

research

∙ 11/23/2017

Improving the Accuracy of Pre-trained Word Embeddings for Sentiment Analysis

Sentiment analysis is one of the well-known tasks and fast growing resea...

0 Seyed Mahdi Rezaeinia, et al. ∙

research

∙ 08/25/2017

Deep Structure for end-to-end inverse rendering

Inverse rendering in a 3D format denoted to recovering the 3D properties...

0 Shima Kamyab, et al. ∙

research

∙ 04/07/2017

Fast Spectral Clustering Using Autoencoders and Landmarks

In this paper, we introduce an algorithm for performing spectral cluster...

0 Ershad Banijamali, et al. ∙

research

∙ 02/10/2017

Generative Mixture of Networks

A generative model based on training deep architectures is proposed. The...

0 Ershad Banijamali, et al. ∙

research

∙ 04/25/2016

Semi-supervised Dictionary Learning Based on Hilbert-Schmidt Independence Criterion

In this paper, a novel semi-supervised dictionary learning and sparse re...

0 Mehrdad J. Gangeh, et al. ∙

research

∙ 03/06/2015

On the Invariance of Dictionary Learning and Sparse Representation to Projecting Data to a Discriminative Space

In this paper, it is proved that dictionary learning and sparse represen...

0 Mehrdad J. Gangeh, et al. ∙

research

∙ 02/20/2015

Supervised Dictionary Learning and Sparse Representation-A Review

Dictionary learning and sparse representation (DLSR) is a recent and suc...

0 Mehrdad J. Gangeh, et al. ∙

research

∙ 12/24/2013

A Fast Greedy Algorithm for Generalized Column Subset Selection

This paper defines a generalized column subset selection problem which i...

0 Ahmed K. Farahat, et al. ∙

research

∙ 07/12/2012

Supervised Texture Classification Using a Novel Compression-Based Similarity Measure

Supervised pixel-based texture classification is usually performed in th...

0 Mehrdad J. Gangeh, et al. ∙

research

∙ 07/10/2012

Kernelized Supervised Dictionary Learning

In this paper, we propose supervised dictionary learning (SDL) by incorp...

0 Mehrdad J. Gangeh, et al. ∙

Ali Ghodsi

Featured Co-authors