BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Ramesh Raskar

Ramesh Raskar appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
Computer VisionCryptography and SecurityMachine Learningcs.CYphysics.opticsDistributed, Parallel, and Cluster ComputingInformation Theorymath.ITArtificial Intelligenceeess.IVGraphicsecon.GNEmerging TechnologiesNetworking and Internet Architecturephysics.soc-phq-fin.ECSocial and Information Networks

Catalog footprint

What is connected

37works
17topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

37 published item(s)

preprint2026arXiv

Imaging Hidden Objects with Consumer LiDAR via Motion Induced Sampling

LiDARs are being increasingly deployed for consumer imaging in handheld, wearable, and robotic applications. These sensors can capture the time-of-flight of light at picosecond resolution, which in principle, enables them to capture information about objects hidden from their field of view. While such non-line-of-sight (NLOS) imaging capabilities have been shown on research-grade LiDARs, they are challenging to achieve on consumer devices due to poor signal quality resulting from low laser power, low spatial resolution, and object and camera motion. Inspired by burst photography and synthetic aperture radar, we propose a multi-frame fusion strategy to overcome these challenges and demonstrate NLOS imaging on consumer LiDAR. We first introduce the motion-induced aperture sampling model to unify the effects of object shape, object motion, and camera motion under a single measurement model. Using this model, we demonstrate several NLOS capabilities on a smartphone-grade LiDAR: (1) 3D reconstruction, (2) single and multi-object tracking, and (3) camera localization using hidden objects. Previously, NLOS imaging capabilities were largely restricted to bulky and expensive research-grade hardware that requires extensive setup and calibration. Our results represent a shift towards plug-and-play NLOS imaging, where anyone can image hidden objects with off-the-shelf hardware ($<100) and no additional setup. We believe that democratization of such capabilities will advance consumer applications of NLOS imaging.

0 citations0 reviewsNo code linkedOpen access
preprint2023arXiv

Physics vs. Learned Priors: Rethinking Camera and Algorithm Design for Task-Specific Imaging

Cameras were originally designed using physics-based heuristics to capture aesthetic images. In recent years, there has been a transformation in camera design from being purely physics-driven to increasingly data-driven and task-specific. In this paper, we present a framework to understand the building blocks of this nascent field of end-to-end design of camera hardware and algorithms. As part of this framework, we show how methods that exploit both physics and data have become prevalent in imaging and computer vision, underscoring a key trend that will continue to dominate the future of task-specific camera design. Finally, we share current barriers to progress in end-to-end design, and hypothesize how these barriers can be overcome.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Challenges of Equitable Vaccine Distribution in the COVID-19 Pandemic

The COVID-19 pandemic has led to a need for widespread and rapid vaccine development. As several vaccines have recently been approved for human use or are in different stages of development, governments across the world are preparing comprehensive guidelines for vaccine distribution and monitoring. In this early article, we identify challenges in logistics, health outcomes, user-centric matters, and communication associated with disease-related, individual, societal, economic, and privacy consequences. Primary challenges include difficulty in equitable distribution, vaccine efficacy, duration of immunity, multi-dose adherence, and privacy-focused record-keeping to be HIPAA compliant. While many of these challenges have been previously identified and addressed, some have not been acknowledged from a comprehensive view accounting for unprecedented interactions between challenges and specific populations. The logistics of equitable widespread vaccine distribution in disparate populations and countries of various economic, racial, and cultural constitutions must be thoroughly examined and accounted for. We also describe unique challenges regarding the efficacy of vaccines in specialized populations including children, the elderly, and immunocompromised individuals. Furthermore, we report the potential for understudied drug-vaccine interactions as well as the possibility that certain vaccine platforms may increase susceptibility to HIV. Given these complicated issues, the importance of privacy-focused, user-centric systems for vaccine education and incentivization along with clear communication from governments, organizations, and academic institutions is imperative. These challenges are by no means insurmountable, but require careful attention to avoid consequences spanning a range of disease-related, individual, societal, economic, and security domains.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Decouple-and-Sample: Protecting sensitive information in task agnostic data release

We propose sanitizer, a framework for secure and task-agnostic data release. While releasing datasets continues to make a big impact in various applications of computer vision, its impact is mostly realized when data sharing is not inhibited by privacy concerns. We alleviate these concerns by sanitizing datasets in a two-stage process. First, we introduce a global decoupling stage for decomposing raw data into sensitive and non-sensitive latent representations. Secondly, we design a local sampling stage to synthetically generate sensitive information with differential privacy and merge it with non-sensitive latent features to create a useful representation while preserving the privacy. This newly formed latent information is a task-agnostic representation of the original dataset with anonymized sensitive information. While most algorithms sanitize data in a task-dependent manner, a few task-agnostic sanitization techniques sanitize data by censoring sensitive information. In this work, we show that a better privacy-utility trade-off is achieved if sensitive information can be synthesized privately. We validate the effectiveness of the sanitizer by outperforming state-of-the-art baselines on the existing benchmark tasks and demonstrating tasks that are not possible using existing techniques.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Fundamentals of Task-Agnostic Data Valuation

We study valuing the data of a data owner/seller for a data seeker/buyer. Data valuation is often carried out for a specific task assuming a particular utility metric, such as test accuracy on a validation set, that may not exist in practice. In this work, we focus on task-agnostic data valuation without any validation requirements. The data buyer has access to a limited amount of data (which could be publicly available) and seeks more data samples from a data seller. We formulate the problem as estimating the differences in the statistical properties of the data at the seller with respect to the baseline data available at the buyer. We capture these statistical differences through second moment by measuring diversity and relevance of the seller's data for the buyer; we estimate these measures through queries to the seller without requesting raw data. We design the queries with the proposed approach so that the seller is blind to the buyer's raw data and has no knowledge to fabricate responses to queries to obtain a desired outcome of the diversity and relevance trade-off.We will show through extensive experiments on real tabular and image datasets that the proposed estimates capture the diversity and relevance of the seller's data for the buyer.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Paper card-based vs application-based vaccine credentials: a comparison

In this early draft, we provide an overview on similarities and differences in the implementation of a paper card-based vaccine credential system and an app-based vaccine credential system. A vaccine credential's primary goal is to regulate entry and ensure safety of individuals within densely packed public locations and workspaces. This is critical for containing the rapid spread of Covid-19 in densely packed public locations since a single individual can infect a large majority of people in a crowd. A vaccine credential can also provide information such as an individual's Covid-19 vaccination history and adverse symptom reaction history to judge their potential impact on the overall health of individuals within densely packed public locations and workspaces. After completing the comparisons, we believe a card-based implementation will benefit regions with less socioeconomic mobility, limited resources, and stagnant administrations. An app-based implementation on the other hand will benefit regions with equitable internet access and lower technological divide. We also believe an interoperable system of both credential systems will work best for regions with enormous working-class populations and dense housing clusters.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Physically Disentangled Representations

State-of-the-art methods in generative representation learning yield semantic disentanglement, but typically do not consider physical scene parameters, such as geometry, albedo, lighting, or camera. We posit that inverse rendering, a way to reverse the rendering process to recover scene parameters from an image, can also be used to learn physically disentangled representations of scenes without supervision. In this paper, we show the utility of inverse rendering in learning representations that yield improved accuracy on downstream clustering, linear classification, and segmentation tasks with the help of our novel Leave-One-Out, Cycle Contrastive loss (LOOCC), which improves disentanglement of scene parameters and robustness to out-of-distribution lighting and viewpoints. We perform a comparison of our method with other generative representation learning methods across a variety of downstream tasks, including face attribute classification, emotion recognition, identification, face segmentation, and car classification. Our physically disentangled representations yield higher accuracy than semantically disentangled alternatives across all tasks and by as much as 18%. We hope that this work will motivate future research in applying advances in inverse rendering and 3D understanding to representation learning.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Splintering with distributions: A stochastic decoy scheme for private computation

Performing computations while maintaining privacy is an important problem in todays distributed machine learning solutions. Consider the following two set ups between a client and a server, where in setup i) the client has a public data vector $\mathbf{x}$, the server has a large private database of data vectors $\mathcal{B}$ and the client wants to find the inner products $\langle \mathbf{x,y_k} \rangle, \forall \mathbf{y_k} \in \mathcal{B}$. The client does not want the server to learn $\mathbf{x}$ while the server does not want the client to learn the records in its database. This is in contrast to another setup ii) where the client would like to perform an operation solely on its data, such as computation of a matrix inverse on its data matrix $\mathbf{M}$, but would like to use the superior computing ability of the server to do so without having to leak $\mathbf{M}$ to the server. \par We present a stochastic scheme for splitting the client data into privatized shares that are transmitted to the server in such settings. The server performs the requested operations on these shares instead of on the raw client data at the server. The obtained intermediate results are sent back to the client where they are assembled by the client to obtain the final result.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Towards Learning Neural Representations from Shadows

We present a method that learns neural shadow fields which are neural scene representations that are only learnt from the shadows present in the scene. While traditional shape-from-shadow (SfS) algorithms reconstruct geometry from shadows, they assume a fixed scanning setup and fail to generalize to complex scenes. Neural rendering algorithms, on the other hand, rely on photometric consistency between RGB images, but largely ignore physical cues such as shadows, which have been shown to provide valuable information about the scene. We observe that shadows are a powerful cue that can constrain neural scene representations to learn SfS, and even outperform NeRF to reconstruct otherwise hidden geometry. We propose a graphics-inspired differentiable approach to render accurate shadows with volumetric rendering, predicting a shadow map that can be compared to the ground truth shadow. Even with just binary shadow maps, we show that neural rendering can localize the object and estimate coarse geometry. Our approach reveals that sparse cues in images can be used to estimate geometry using differentiable volumetric rendering. Moreover, our framework is highly generalizable and can work alongside existing 3D reconstruction techniques that otherwise only use photometric consistency.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Visual Transformer Meets CutMix for Improved Accuracy, Communication Efficiency, and Data Privacy in Split Learning

This article seeks for a distributed learning solution for the visual transformer (ViT) architectures. Compared to convolutional neural network (CNN) architectures, ViTs often have larger model sizes, and are computationally expensive, making federated learning (FL) ill-suited. Split learning (SL) can detour this problem by splitting a model and communicating the hidden representations at the split-layer, also known as smashed data. Notwithstanding, the smashed data of ViT are as large as and as similar as the input data, negating the communication efficiency of SL while violating data privacy. To resolve these issues, we propose a new form of CutSmashed data by randomly punching and compressing the original smashed data. Leveraging this, we develop a novel SL framework for ViT, coined CutMixSL, communicating CutSmashed data. CutMixSL not only reduces communication costs and privacy leakage, but also inherently involves the CutMix data augmentation, improving accuracy and scalability. Simulations corroborate that CutMixSL outperforms baselines such as parallelized SL and SplitFed that integrates FL with SL.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Advances and Open Problems in Federated Learning

Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a central server (e.g. service provider), while keeping the training data decentralized. FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches. Motivated by the explosive growth in FL research, this paper discusses recent advances and presents an extensive collection of open problems and challenges.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

MIT SafePaths Card (MiSaCa): Augmenting Paper Based Vaccination Cards with Printed Codes

In this early draft, we describe a user-centric, card-based system for vaccine distribution. Our system makes use of digitally signed QR codes and their use for phased vaccine distribution, vaccine administration/record-keeping, immunization verification, and follow-up symptom reporting. Furthermore, we propose and describe a complementary scanner app system to be used by vaccination clinics, public health officials, and immunization verification parties to effectively utilize card-based framework. We believe that the proposed system provides a privacy-preserving and efficient framework for vaccine distribution in both developed and developing regions.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Safepaths: Vaccine Diary Protocol and Decentralized Vaccine Coordination System using a Privacy Preserving User Centric Experience

In this early draft, we present an end-to-end decentralized protocol for the secure and privacy preserving workflow of vaccination, vaccination status verification, and adverse reactions or symptoms reporting. The proposed system improves the efficiency, privacy, equity, and effectiveness of the existing manual system while remaining interoperable with its capabilities. We also discuss various security concerns and alternate methodologies based on the proposed protocols.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Spatial K-anonymity: A Privacy-preserving Method for COVID-19 Related Geospatial Technologies

There is a growing need for spatial privacy considerations in the many geo-spatial technologies that have been created as solutions for COVID-19-related issues. Although effective geo-spatial technologies have already been rolled out, most have significantly sacrificed privacy for utility. In this paper, we explore spatial k-anonymity, a privacy-preserving method that can address this unnecessary tradeoff by providing the best of both privacy and utility. After evaluating its past implications in geo-spatial use cases, we propose applications of spatial k-anonymity in the data sharing and managing of COVID-19 contact tracing technologies as well as heat maps showing a user's travel history. We then justify our propositions by comparing spatial k-anonymity with several other spatial privacy methods, including differential privacy, geo-indistinguishability, and manual consent based redaction. Our hope is to raise awareness of the ever-growing risks associated with spatial privacy and how they can be solved with Spatial K-anonymity.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Adding Location and Global Context to the Google/Apple Exposure Notification Bluetooth API

Contact tracing requires a strong understanding of the context of a user, and location with other sensory data could provide a context for any infection encounter. Although Bluetooth technology gives a good insight into the proximity aspect of an encounter, it does not provide any location context related to it which helps to make better decisions. Using the ideas presented in this paper, one shall be able to obtain this valuable information that could address the problem of false-positive and false-negative to a certain extent. All of this within the purview of Google/Apple Exposure Notification (GAEN) specification, while preserving complete user privacy. There are four ways of propagating context between any two users. Two such methods allow private location logging, without revealing the location history within an app. The other two are encryption-based methods. The first encryption method is a variant of Apple's FindMy protocol, that allows nearby Apple devices to capture the GPS location of a lost Apple device. The second encryption is a minor modification of the existing GAEN protocol so that global context is available to a healthy phone only when it is exposed - this is a better option comparatively. It will still be the role of Public Health smartphone app to decide, on how to use the location-time context, to build a full-fledged contact tracing and public health solution. Lastly, we highlight the benefits and potential privacy issues with each of these context propagation methods proposed here.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Automatic Differentiation for All Photons Imaging to See Inside Volumetric Scattering Media

Imaging through dense scattering media - such as biological tissue, fog, and smoke - has applications in the medical and robotics fields. We propose a new framework using automatic differentiation for All Photons Imaging through homogeneous scattering media with unknown optical properties for non-invasive sensing and diagnostics. We overcome the need for the imaging target to be visible to the illumination source in All Photons Imaging, enabling practical and non-invasive imaging through turbid media with a simple optical setup. Our method does not require calibration to acquire the sensor position or optical properties of the media.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Bluetooth based Proximity, Multi-hop Analysis and Bi-directional Trust: Epidemics and More

In this paper, we propose a trust layer on top of Bluetooth and similar wireless communication technologies that can form mesh networks. This layer as a protocol enables computing trust scores based on proximity and bi-directional transfer of messages in multiple hops across a network of mobile devices. We describe factors and an approach for determining these trust scores and highlight its applications during epidemics such as COVID-19 through improved contact-tracing, better privacy and verification for sensitive data sharing in the numerous Bluetooth and GPS based mobile applications that are being developed to track the spread.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Comparing manual contact tracing and digital contact advice

Manual contact tracing is a top-down solution that starts with contact tracers at the public health level, who identify the contacts of infected individuals, interview them to get additional context about the exposure, and also monitor their symptoms and support them until the incubation period is passed. On the other hand, digital contact tracing is a bottom-up solution that starts with citizens who on obtaining a notification about possible exposure to an infected individual may choose to ignore the notification, get tested to determine if they were actually exposed or self-isolate and monitor their symptoms over the next two weeks. Most experts recommend a combination of manual contact tracing and digital contact advice but they are not based on a scientific basis. For example, a possible hybrid solution could involve a smartphone based alert that requests the possible contact of an infected individual to call the Public Health (PH) number for next steps, or in some cases, suggest ways to self-assess in order to reduce the burden on PH so only most critical cases require a phone conversation. In this paper, we aim to compare the manual and digital approaches to contact tracing and provide suggestions for potential hybrid solutions.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

COVID-19 Contact-Tracing Mobile Apps: Evaluation and Assessment for Decision Makers

A number of groups, from governments to non-profits, have quickly acted to innovate the contact-tracing process: they are designing, building, and launching contact-tracing apps in response to the COVID-19 crisis. A diverse range of approaches exist, creating challenging choices for officials looking to implement contact-tracing technology in their community and raising concerns about these choices among citizens asked to participate in contact tracing. We are frequently asked how to evaluate and differentiate between the options for contact-tracing applications. Here, we share the questions we ask about app features and plans when reviewing the many contact-tracing apps appearing on the global stage.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

NoPeek: Information leakage reduction to share activations in distributed deep learning

For distributed machine learning with sensitive data, we demonstrate how minimizing distance correlation between raw data and intermediary representations reduces leakage of sensitive raw data patterns across client communications while maintaining model accuracy. Leakage (measured using distance correlation between input and intermediate representations) is the risk associated with the invertibility of raw data from intermediary representations. This can prevent client entities that hold sensitive data from using distributed deep learning services. We demonstrate that our method is resilient to such reconstruction attacks and is based on reduction of distance correlation between raw data and learned representations during training and inference with image datasets. We prevent such reconstruction of raw data while maintaining information required to sustain good classification accuracies.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

PPContactTracing: A Privacy-Preserving Contact Tracing Protocol for COVID-19 Pandemic

Several contact tracing solutions have been proposed and implemented all around the globe to combat the spread of COVID-19 pandemic. But, most of these solutions endanger the privacy rights of the individuals and hinder their widespread adoption. We propose a privacy-preserving contact tracing protocol for the efficient tracing of the spread of the global pandemic. It is based on the private set intersection (PSI) protocol and utilizes the homomorphic properties to preserve the privacy at the individual level. A hierarchical model for the representation of landscapes and rate-limiting factor on the number of queries have been adopted to maintain the efficiency of the protocol.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Privacy Guidelines for Contact Tracing Applications

Contact tracing is a very powerful method to implement and enforce social distancing to avoid spreading of infectious diseases. The traditional approach of contact tracing is time consuming, manpower intensive, dangerous and prone to error due to fatigue or lack of skill. Due to this there is an emergence of mobile based applications for contact tracing. These applications primarily utilize a combination of GPS based absolute location and Bluetooth based relative location remitted from user's smartphone to infer various insights. These applications have eased the task of contact tracing; however, they also have severe implication on user's privacy, for example, mass surveillance, personal information leakage and additionally revealing the behavioral patterns of the user. This impact on user's privacy leads to trust deficit in these applications, and hence defeats their purpose. In this work we discuss the various scenarios which a contact tracing application should be able to handle. We highlight the privacy handling of some of the prominent contact tracing applications. Additionally, we describe the various threat actors who can disrupt its working, or misuse end user's data, or hamper its mass adoption. Finally, we present privacy guidelines for contact tracing applications from different stakeholder's perspective. To best of our knowledge, this is the first generic work which provides privacy guidelines for contact tracing applications.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

SplitNN-driven Vertical Partitioning

In this work, we introduce SplitNN-driven Vertical Partitioning, a configuration of a distributed deep learning method called SplitNN to facilitate learning from vertically distributed features. SplitNN does not share raw data or model details with collaborating institutions. The proposed configuration allows training among institutions holding diverse sources of data without the need of complex encryption algorithms or secure computation protocols. We evaluate several configurations to merge the outputs of the split models, and compare performance and resource efficiency. The method is flexible and allows many different configurations to tackle the specific challenges posed by vertically split datasets.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Deep Learning the City : Quantifying Urban Perception At A Global Scale

Computer vision methods that quantify the perception of urban environment are increasingly being used to study the relationship between a city's physical appearance and the behavior and health of its residents. Yet, the throughput of current methods is too limited to quantify the perception of cities across the world. To tackle this challenge, we introduce a new crowdsourced dataset containing 110,988 images from 56 cities, and 1,170,000 pairwise comparisons provided by 81,630 online volunteers along six perceptual attributes: safe, lively, boring, wealthy, depressing, and beautiful. Using this data, we train a Siamese-like convolutional neural architecture, which learns from a joint classification and ranking loss, to predict human judgments of pairwise image comparisons. Our results show that crowdsourcing combined with neural networks can produce urban perception data at the global scale.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Deep video gesture recognition using illumination invariants

In this paper we present architectures based on deep neural nets for gesture recognition in videos, which are invariant to local scaling. We amalgamate autoencoder and predictor architectures using an adaptive weighting scheme coping with a reduced size labeled dataset, while enriching our models from enormous unlabeled sets. We further improve robustness to lighting conditions by introducing a new adaptive filer based on temporal local scale normalization. We provide superior results over known methods, including recent reported approaches based on neural nets.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Multi-velocity neural networks for gesture recognition in videos

We present a new action recognition deep neural network which adaptively learns the best action velocities in addition to the classification. While deep neural networks have reached maturity for image understanding tasks, we are still exploring network topologies and features to handle the richer environment of video clips. Here, we tackle the problem of multiple velocities in action recognition, and provide state-of-the-art results for gesture recognition, on known and new collected datasets. We further provide the training steps for our semi-supervised network, suited to learn from huge unlabeled datasets with only a fraction of labeled examples.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Sweep Distortion Removal from THz Images via Blind Demodulation

Heavy sweep distortion induced by alignments and inter-reflections of layers of a sample is a major burden in recovering 2D and 3D information in time resolved spectral imaging. This problem cannot be addressed by conventional denoising and signal processing techniques as it heavily depends on the physics of the acquisition. Here we propose and implement an algorithmic framework based on low-rank matrix recovery and alternating minimization that exploits the forward model for THz acquisition. The method allows recovering the original signal in spite of the presence of temporal-spatial distortions. We address a blind-demodulation problem, where based on several observations of the sample texture modulated by an undesired sweep pattern, the two classes of signals are separated. The performance of the method is examined in both synthetic and experimental data, and the successful reconstructions are demonstrated. The proposed general scheme can be implemented to advance inspection and imaging applications in THz and other time-resolved sensing modalities.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

A Light Transport Model for Mitigating Multipath Interference in TOF Sensors

Continuous-wave Time-of-flight (TOF) range imaging has become a commercially viable technology with many applications in computer vision and graphics. However, the depth images obtained from TOF cameras contain scene dependent errors due to multipath interference (MPI). Specifically, MPI occurs when multiple optical reflections return to a single spatial location on the imaging sensor. Many prior approaches to rectifying MPI rely on sparsity in optical reflections, which is an extreme simplification. In this paper, we correct MPI by combining the standard measurements from a TOF camera with information from direct and global light transport. We report results on both simulated experiments and physical experiments (using the Kinect sensor). Our results, evaluated against ground truth, demonstrate a quantitative improvement in depth accuracy.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Coreset-Based Adaptive Tracking

We propose a method for learning from streaming visual data using a compact, constant size representation of all the data that was seen until a given moment. Specifically, we construct a 'coreset' representation of streaming data using a parallelized algorithm, which is an approximation of a set with relation to the squared distances between this set and all other points in its ambient space. We learn an adaptive object appearance model from the coreset tree in constant time and logarithmic space and use it for object tracking by detection. Our method obtains excellent results for object tracking on three standard datasets over more than 100 videos. The ability to summarize data efficiently makes our method ideally suited for tracking in long videos in presence of space and time constraints. We demonstrate this ability by outperforming a variety of algorithms on the TLD dataset with 2685 frames on average. This coreset based learning approach can be applied for both real-time learning of small, varied data and fast learning of big data.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Frequency Domain TOF: Encoding Object Depth in Modulation Frequency

Time of flight cameras may emerge as the 3-D sensor of choice. Today, time of flight sensors use phase-based sampling, where the phase delay between emitted and received, high-frequency signals encodes distance. In this paper, we present a new time of flight architecture that relies only on frequency---we refer to this technique as frequency-domain time of flight (FD-TOF). Inspired by optical coherence tomography (OCT), FD-TOF excels when frequency bandwidth is high. With the increasing frequency of TOF sensors, new challenges to time of flight sensing continue to emerge. At high frequencies, FD-TOF offers several potential benefits over phase-based time of flight methods.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Super-Resolution in Phase Space

This work considers the problem of super-resolution. The goal is to resolve a Dirac distribution from knowledge of its discrete, low-pass, Fourier measurements. Classically, such problems have been dealt with parameter estimation methods. Recently, it has been shown that convex-optimization based formulations facilitate a continuous time solution to the super-resolution problem. Here we treat super-resolution from low-pass measurements in Phase Space. The Phase Space transformation parametrically generalizes a number of well known unitary mappings such as the Fractional Fourier, Fresnel, Laplace and Fourier transforms. Consequently, our work provides a general super- resolution strategy which is backward compatible with the usual Fourier domain result. We consider low-pass measurements of Dirac distributions in Phase Space and show that the super-resolution problem can be cast as Total Variation minimization. Remarkably, even though are setting is quite general, the bounds on the minimum separation distance of Dirac distributions is comparable to existing methods.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

A Compressive Multi-Mode Superresolution Display

Compressive displays are an emerging technology exploring the co-design of new optical device configurations and compressive computation. Previously, research has shown how to improve the dynamic range of displays and facilitate high-quality light field or glasses-free 3D image synthesis. In this paper, we introduce a new multi-mode compressive display architecture that supports switching between 3D and high dynamic range (HDR) modes as well as a new super-resolution mode. The proposed hardware consists of readily-available components and is driven by a novel splitting algorithm that computes the pixel states from a target high-resolution image. In effect, the display pixels present a compressed representation of the target image that is perceived as a single, high resolution image.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Resolving Multi-path Interference in Time-of-Flight Imaging via Modulation Frequency Diversity and Sparse Regularization

Time-of-flight (ToF) cameras calculate depth maps by reconstructing phase shifts of amplitude-modulated signals. For broad illumination or transparent objects, reflections from multiple scene points can illuminate a given pixel, giving rise to an erroneous depth map. We report here a sparsity regularized solution that separates K-interfering components using multiple modulation frequency measurements. The method maps ToF imaging to the general framework of spectral estimation theory and has applications in improving depth profiles and exploiting multiple scattering.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Reconstruction of hidden 3D shapes using diffuse reflections

We analyze multi-bounce propagation of light in an unknown hidden volume and demonstrate that the reflected light contains sufficient information to recover the 3D structure of the hidden scene. We formulate the forward and inverse theory of secondary and tertiary scattering reflection using ideas from energy front propagation and tomography. We show that using careful choice of approximations, such as Fresnel approximation, greatly simplifies this problem and the inversion can be achieved via a backpropagation process. We provide a theoretical analysis of the invertibility, uniqueness and choices of space-time-angle dimensions using synthetic examples. We show that a 2D streak camera can be used to discover and reconstruct hidden geometry. Using a 1D high speed time of flight camera, we show that our method can be used recover 3D shapes of objects "around the corner".

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Progressive versus Random Projections for Compressive Capture of Images, Lightfields and Higher Dimensional Visual Signals

Computational photography involves sophisticated capture methods. A new trend is to capture projection of higher dimensional visual signals such as videos, multi-spectral data and lightfields on lower dimensional sensors. Carefully designed capture methods exploit the sparsity of the underlying signal in a transformed domain to reduce the number of measurements and use an appropriate reconstruction method. Traditional progressive methods may capture successively more detail using a sequence of simple projection basis, such as DCT or wavelets and employ straightforward backprojection for reconstruction. Randomized projection methods do not use any specific sequence and use L0 minimization for reconstruction. In this paper, we analyze the statistical properties of natural images, videos, multi-spectral data and light-fields and compare the effectiveness of progressive and random projections. We define effectiveness by plotting reconstruction SNR against compression factor. The key idea is a procedure to measure best-case effectiveness that is fast, independent of specific hardware and independent of the reconstruction procedure. We believe this is the first empirical study to compare different lossy capture strategies without the complication of hardware or reconstruction ambiguity. The scope is limited to linear non-adaptive sensing. The results show that random projections produce significant advantages over other projections only for higher dimensional signals, and suggest more research to nascent adaptive and non-linear projection methods.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Ray-Based Reflectance Model for Diffraction

We present a novel method of simulating wave effects in graphics using ray--based renderers with a new function: the Wave BSDF (Bidirectional Scattering Distribution Function). Reflections from neighboring surface patches represented by local BSDFs are mutually independent. However, in many surfaces with wavelength-scale microstructures, interference and diffraction requires a joint analysis of reflected wavefronts from neighboring patches. We demonstrate a simple method to compute the BSDF for the entire microstructure, which can be used independently for each patch. This allows us to use traditional ray--based rendering pipelines to synthesize wave effects of light and sound. We exploit the Wigner Distribution Function (WDF) to create transmissive, reflective, and emissive BSDFs for various diffraction phenomena in a physically accurate way. In contrast to previous methods for computing interference, we circumvent the need to explicitly keep track of the phase of the wave by using BSDFs that include positive as well as negative coefficients. We describe and compare the theory in relation to well understood concepts in rendering and demonstrate a straightforward implementation. In conjunction with standard raytracers, such as PBRT, we demonstrate wave effects for a range of scenarios such as multi--bounce diffraction materials, holograms and reflection of high frequency surfaces.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

View-Dependent Displays and the Space of Light Fields

In this paper we explore how light propagates from thin elements into a volume for viewing. In particular, devices that are typically connected with geometric optics, like parallax barriers, differ in treatment with those that obey physical optics, like holograms. However, the two concepts are often used to achieve the same effect of capturing or displaying a combination of spatial and angular information. This paper attempts to connect the two approaches under a general framework based in ray space, from which insights into applications and limitations of both parallax-based and holography-based systems can be observed. We show that each display form can generate a light distribution that can always be expressed as a rank-1 matrix. Knowledge of this limitation is then discussed in the context of extending the capabilities of current display forms by considering the use of partially coherent light.

0 citations0 reviewsNo code linkedOpen access