
Multimodal Prediction and Personalization of Photo Edits with Deep Generative Models
Professionalgrade software applications are powerful but complicatedexpert users can achieve impressive results, but novices often struggle to complete even basic tasks. Photo editing is a prime example: after loading a photo, the user is confronted with an array of cryptic sliders like "clarity", "temp", and "highlights". An automatically generated suggestion could help, but there is no single "correct" edit for a given imagedifferent experts may make very different aesthetic decisions when faced with the same image, and a single expert may make different choices depending on the intended use of the image (or on a whim). We therefore want a system that can propose multiple diverse, highquality edits while also learning from and adapting to a user's aesthetic preferences. In this work, we develop a statistical model that meets these objectives. Our model builds on recent advances in neural network generative modeling and scalable inference, and uses hierarchical structure to learn editing patterns across many diverse users. Empirically, we find that our model outperforms other approaches on this challenging multimodal prediction task.
04/17/2017 ∙ by Ardavan Saeedi, et al. ∙ 0 ∙ shareread it

Deep Successor Reinforcement Learning
Learning robust value functions given raw observations and rewards is now possible with modelfree and modelbased deep reinforcement learning algorithms. There is a third alternative, called Successor Representations (SR), which decomposes the value function into two components  a reward predictor and a successor map. The successor map represents the expected future state occupancy from any given state and the reward predictor maps states to scalar rewards. The value function of a state can be computed as the inner product between the successor map and the reward weights. In this paper, we present DSR, which generalizes SR within an endtoend deep reinforcement learning framework. DSR has several appealing properties including: increased sensitivity to distal reward changes due to factorization of reward and world dynamics, and the ability to extract bottleneck states (subgoals) given successor maps trained under a random policy. We show the efficacy of our approach on two diverse environments given raw pixel observations  simple gridworld domains (MazeBase) and the Doom game engine.
06/08/2016 ∙ by Tejas D. Kulkarni, et al. ∙ 0 ∙ shareread it

The Segmented iHMM: A Simple, Efficient Hierarchical Infinite HMM
We propose the segmented iHMM (siHMM), a hierarchical infinite hidden Markov model (iHMM) that supports a simple, efficient inference scheme. The siHMM is well suited to segmentation problems, where the goal is to identify points at which a time series transitions from one relatively stable regime to a new regime. Conventional iHMMs often struggle with such problems, since they have no mechanism for distinguishing between high and lowlevel dynamics. Hierarchical HMMs (HHMMs) can do better, but they require much more complex and expensive inference algorithms. The siHMM retains the simplicity and efficiency of the iHMM, but outperforms it on a variety of segmentation problems, achieving performance that matches or exceeds that of a more complicated HHMM.
02/20/2016 ∙ by Ardavan Saeedi, et al. ∙ 0 ∙ shareread it

Automatic Inference for Inverting Software Simulators via Probabilistic Programming
Models of complex systems are often formalized as sequential software simulators: computationally intensive programs that iteratively build up probable system configurations given parameters and initial conditions. These simulators enable modelers to capture effects that are difficult to characterize analytically or summarize statistically. However, in many realworld applications, these simulations need to be inverted to match the observed data. This typically requires the custom design, derivation and implementation of sophisticated inversion algorithms. Here we give a framework for inverting a broad class of complex software simulators via probabilistic programming and automatic inference, using under 20 lines of probabilistic code. Our approach is based on a formulation of inversion as approximate inference in a simple sequential probabilistic model. We implement four inference strategies, including MetropolisHastings, a sequentialized MetropolisHastings scheme, and a particle Markov chain Monte Carlo scheme, requiring 4 or fewer lines of probabilistic code each. We demonstrate our framework by applying it to invert a real geological software simulator from the oil and gas industry.
05/31/2015 ∙ by Ardavan Saeedi, et al. ∙ 0 ∙ shareread it

JUMPMeans: SmallVariance Asymptotics for Markov Jump Processes
Markov jump processes (MJPs) are used to model a wide range of phenomena from disease progression to RNA path folding. However, maximum likelihood estimation of parametric models leads to degenerate trajectories and inferential performance is poor in nonparametric models. We take a smallvariance asymptotics (SVA) approach to overcome these limitations. We derive the smallvariance asymptotics for parametric and nonparametric MJPs for both directly observed and hidden state models. In the parametric case we obtain a novel objective function which leads to nondegenerate trajectories. To derive the nonparametric version we introduce the gammagamma process, a novel extension to the gammaexponential process. We propose algorithms for each of these formulations, which we call JUMPmeans. Our experiments demonstrate that JUMPmeans is competitive with or outperforms widely used MJP inference approaches in terms of both speed and reconstruction accuracy.
03/01/2015 ∙ by Jonathan H. Huggins, et al. ∙ 0 ∙ shareread it

Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
Learning goaldirected behavior in environments with sparse feedback is a major challenge for reinforcement learning algorithms. The primary difficulty arises due to insufficient exploration, resulting in an agent being unable to learn robust value functions. Intrinsically motivated agents can explore new behavior for its own sake rather than to directly solve problems. Such intrinsic behaviors could eventually help the agent solve tasks posed by the environment. We present hierarchicalDQN (hDQN), a framework to integrate hierarchical value functions, operating at different temporal scales, with intrinsically motivated deep reinforcement learning. A toplevel value function learns a policy over intrinsic goals, and a lowerlevel function learns a policy over atomic actions to satisfy the given goals. hDQN allows for flexible goal specifications, such as functions over entities and relations. This provides an efficient space for exploration in complicated environments. We demonstrate the strength of our approach on two problems with very sparse, delayed feedback: (1) a complex discrete stochastic decision process, and (2) the classic ATARI game `Montezuma's Revenge'.
04/20/2016 ∙ by Tejas D. Kulkarni, et al. ∙ 0 ∙ shareread it

Detailed Derivations of SmallVariance Asymptotics for some Hierarchical Bayesian Nonparametric Models
In this note we provide detailed derivations of two versions of smallvariance asymptotics for hierarchical Dirichlet process (HDP) mixture models and the HDP hidden Markov model (HDPHMM, a.k.a. the infinite HMM). We include derivations for the probabilities of certain CRP and CRF partitions, which are of more general interest.
12/31/2014 ∙ by Jonathan H. Huggins, et al. ∙ 0 ∙ shareread it

Variational Particle Approximations
Approximate inference in highdimensional, discrete probabilistic models is a central problem in computational statistics and machine learning. This paper describes discrete particle variational inference (DPVI), a new approach that combines key strengths of Monte Carlo, variational and searchbased techniques. DPVI is based on a novel family of particlebased variational approximations that can be fit using simple, fast, deterministic search techniques. Like Monte Carlo, DPVI can handle multiple modes, and yields exact results in a welldefined limit. Like unstructured meanfield, DPVI is based on optimizing a lower bound on the partition function; when this quantity is not of intrinsic interest, it facilitates convergence assessment and debugging. Like both Monte Carlo and combinatorial search, DPVI can take advantage of factorization, sequential structure, and custom search operators. This paper defines DPVI particlebased approximation family and partition function lower bounds, along with the sequential DPVI and local DPVI algorithm templates for optimizing them. DPVI is illustrated and evaluated via experiments on lattice Markov Random Fields, nonparametric Bayesian mixtures and blockmodels, and parametric as well as nonparametric hidden Markov models. Results include applications to realworld spikesorting and relational modeling problems, and show that DPVI can offer appealing time/accuracy tradeoffs as compared to multiple alternatives.
02/24/2014 ∙ by Ardavan Saeedi, et al. ∙ 0 ∙ shareread it

Nonparametric Spherical Topic Modeling with Word Embeddings
Traditional topic models do not account for semantic regularities in language. Recent distributional representations of words exhibit semantic consistency over directional metrics such as cosine similarity. However, neither categorical nor Gaussian observational distributions used in existing topic models are appropriate to leverage such correlations. In this paper, we propose to use the von MisesFisher distribution to model the density of words over a unit sphere. Such a representation is wellsuited for directional data. We use a Hierarchical Dirichlet Process for our base topic model and propose an efficient inference algorithm based on Stochastic Variational Inference. This model enables us to naturally exploit the semantic structures of word embeddings while flexibly discovering the number of topics. Experiments demonstrate that our method outperforms competitive approaches in terms of topic coherence on two different text corpora while offering efficient inference.
04/01/2016 ∙ by Kayhan Batmanghelich, et al. ∙ 0 ∙ shareread it

Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion
The predictive performance of supervised learning algorithms depends on the quality of labels. In a typical label collection process, multiple annotators provide subjective noisy estimates of the "truth" under the influence of their varying skilllevels and biases. Blindly treating these noisy labels as the ground truth limits the accuracy of learning algorithms in the presence of strong disagreement. This problem is critical for applications in domains such as medical imaging where both the annotation cost and interobserver variability are high. In this work, we present a method for simultaneously learning the individual annotator model and the underlying true label distribution, using only noisy observations. Each annotator is modeled by a confusion matrix that is jointly estimated along with the classifier predictions. We propose to add a regularization term to the loss function that encourages convergence to the true annotator confusion matrix. We provide a theoretical argument as to how the regularization is essential to our approach both for the case of single annotator and multiple annotators. Despite the simplicity of the idea, experiments on image classification tasks with both simulated and real labels show that our method either outperforms or performs on par with the stateoftheart methods and is capable of estimating the skills of annotators even with a single label available per image.
02/10/2019 ∙ by Ryutaro Tanno, et al. ∙ 0 ∙ shareread it
Ardavan Saeedi
is this you? claim profile