Matching Items (16)
Filtering by
- All Subjects: deep learning
- Creators: Turaga, Pavan
Description
Light field imaging is limited in its computational processing demands of high
sampling for both spatial and angular dimensions. Single-shot light field cameras
sacrifice spatial resolution to sample angular viewpoints, typically by multiplexing
incoming rays onto a 2D sensor array. While this resolution can be recovered using
compressive sensing, these iterative solutions are slow in processing a light field. We
present a deep learning approach using a new, two branch network architecture,
consisting jointly of an autoencoder and a 4D CNN, to recover a high resolution
4D light field from a single coded 2D image. This network decreases reconstruction
time significantly while achieving average PSNR values of 26-32 dB on a variety of
light fields. In particular, reconstruction time is decreased from 35 minutes to 6.7
minutes as compared to the dictionary method for equivalent visual quality. These
reconstructions are performed at small sampling/compression ratios as low as 8%,
allowing for cheaper coded light field cameras. We test our network reconstructions
on synthetic light fields, simulated coded measurements of real light fields captured
from a Lytro Illum camera, and real coded images from a custom CMOS diffractive
light field camera. The combination of compressive light field capture with deep
learning allows the potential for real-time light field video acquisition systems in the
future.
sampling for both spatial and angular dimensions. Single-shot light field cameras
sacrifice spatial resolution to sample angular viewpoints, typically by multiplexing
incoming rays onto a 2D sensor array. While this resolution can be recovered using
compressive sensing, these iterative solutions are slow in processing a light field. We
present a deep learning approach using a new, two branch network architecture,
consisting jointly of an autoencoder and a 4D CNN, to recover a high resolution
4D light field from a single coded 2D image. This network decreases reconstruction
time significantly while achieving average PSNR values of 26-32 dB on a variety of
light fields. In particular, reconstruction time is decreased from 35 minutes to 6.7
minutes as compared to the dictionary method for equivalent visual quality. These
reconstructions are performed at small sampling/compression ratios as low as 8%,
allowing for cheaper coded light field cameras. We test our network reconstructions
on synthetic light fields, simulated coded measurements of real light fields captured
from a Lytro Illum camera, and real coded images from a custom CMOS diffractive
light field camera. The combination of compressive light field capture with deep
learning allows the potential for real-time light field video acquisition systems in the
future.
ContributorsGupta, Mayank (Author) / Turaga, Pavan (Thesis advisor) / Yang, Yezhou (Committee member) / Li, Baoxin (Committee member) / Arizona State University (Publisher)
Created2017
Description
In recent years, the widespread use of deep neural networks (DNNs) has facilitated great improvements in performance for computer vision tasks like image classification and object recognition. In most realistic computer vision applications, an input image undergoes some form of image distortion such as blur and additive noise during image acquisition or transmission. Deep networks trained on pristine images perform poorly when tested on such distortions. DNN predictions have also been shown to be vulnerable to carefully crafted adversarial perturbations. Specifically, so-called universal adversarial perturbations are image-agnostic perturbations that can be added to any image and can fool a target network into making erroneous predictions. This work proposes selective DNN feature regeneration to improve the robustness of existing DNNs to image distortions and universal adversarial perturbations.
In the context of common naturally occurring image distortions, a metric is proposed to identify the most susceptible DNN convolutional filters and rank them in order of the highest gain in classification accuracy upon correction. The proposed approach called DeepCorrect applies small stacks of convolutional layers with residual connections at the output of these ranked filters and trains them to correct the most distortion-affected filter activations, whilst leaving the rest of the pre-trained filter outputs in the network unchanged. Performance results show that applying DeepCorrect models for common vision tasks significantly improves the robustness of DNNs against distorted images and outperforms other alternative approaches.
In the context of universal adversarial perturbations, departing from existing defense strategies that work mostly in the image domain, a novel and effective defense which only operates in the DNN feature domain is presented. This approach identifies pre-trained convolutional features that are most vulnerable to adversarial perturbations and deploys trainable feature regeneration units which transform these DNN filter activations into resilient features that are robust to universal perturbations. Regenerating only the top 50% adversarially susceptible activations in at most 6 DNN layers and leaving all remaining DNN activations unchanged can outperform existing defense strategies across different network architectures and across various universal attacks.
In the context of common naturally occurring image distortions, a metric is proposed to identify the most susceptible DNN convolutional filters and rank them in order of the highest gain in classification accuracy upon correction. The proposed approach called DeepCorrect applies small stacks of convolutional layers with residual connections at the output of these ranked filters and trains them to correct the most distortion-affected filter activations, whilst leaving the rest of the pre-trained filter outputs in the network unchanged. Performance results show that applying DeepCorrect models for common vision tasks significantly improves the robustness of DNNs against distorted images and outperforms other alternative approaches.
In the context of universal adversarial perturbations, departing from existing defense strategies that work mostly in the image domain, a novel and effective defense which only operates in the DNN feature domain is presented. This approach identifies pre-trained convolutional features that are most vulnerable to adversarial perturbations and deploys trainable feature regeneration units which transform these DNN filter activations into resilient features that are robust to universal perturbations. Regenerating only the top 50% adversarially susceptible activations in at most 6 DNN layers and leaving all remaining DNN activations unchanged can outperform existing defense strategies across different network architectures and across various universal attacks.
ContributorsBorkar, Tejas Shyam (Author) / Karam, Lina J (Thesis advisor) / Turaga, Pavan (Committee member) / Jayasuriya, Suren (Committee member) / Tepedelenlioğlu, Cihan (Committee member) / Arizona State University (Publisher)
Created2020
Description
The autonomous vehicle technology has come a long way, but currently, there are no companies that are able to offer fully autonomous ride in any conditions, on any road without any human supervision. These systems should be extensively trained and validated to guarantee safe human transportation. Any small errors in the system functionality may lead to fatal accidents and may endanger human lives. Deep learning methods are widely used for environment perception and prediction of hazardous situations. These techniques require huge amount of training data with both normal and abnormal samples to enable the vehicle to avoid a dangerous situation.
The goal of this thesis is to generate simulations from real-world tricky collision scenarios for training and testing autonomous vehicles. Dashcam crash videos from the internet can now be utilized to extract valuable collision data and recreate the crash scenarios in a simulator. The problem of extracting 3D vehicle trajectories from videos recorded by an unknown monocular camera source is solved using a modular approach. The framework is divided into two stages: (a) extracting meaningful adversarial trajectories from short crash videos, and (b) developing methods to automatically process and simulate the vehicle trajectories on a vehicle simulator.
The goal of this thesis is to generate simulations from real-world tricky collision scenarios for training and testing autonomous vehicles. Dashcam crash videos from the internet can now be utilized to extract valuable collision data and recreate the crash scenarios in a simulator. The problem of extracting 3D vehicle trajectories from videos recorded by an unknown monocular camera source is solved using a modular approach. The framework is divided into two stages: (a) extracting meaningful adversarial trajectories from short crash videos, and (b) developing methods to automatically process and simulate the vehicle trajectories on a vehicle simulator.
ContributorsBashetty, Sai Krishna (Author) / Fainkeos, Georgios (Thesis advisor) / Amor, Heni Ben (Thesis advisor) / Turaga, Pavan (Committee member) / Arizona State University (Publisher)
Created2019
Description
Over the last decade, deep neural networks also known as deep learning, combined with large databases and specialized hardware for computation, have made major strides in important areas such as computer vision, computational imaging and natural language processing. However, such frameworks currently suffer from some drawbacks. For example, it is generally not clear how the architectures are to be designed for different applications, or how the neural networks behave under different input perturbations and it is not easy to make the internal representations and parameters more interpretable. In this dissertation, I propose building constraints into feature maps, parameters and and design of algorithms involving neural networks for applications in low-level vision problems such as compressive imaging and multi-spectral image fusion, and high-level inference problems including activity and face recognition. Depending on the application, such constraints can be used to design architectures which are invariant/robust to certain nuisance factors, more efficient and, in some cases, more interpretable. Through extensive experiments on real-world datasets, I demonstrate these advantages of the proposed methods over conventional frameworks.
ContributorsLohit, Suhas Anand (Author) / Turaga, Pavan (Thesis advisor) / Spanias, Andreas (Committee member) / Li, Baoxin (Committee member) / Jayasuriya, Suren (Committee member) / Arizona State University (Publisher)
Created2019
Description
The recent increase in users of cellular networks necessitates the use of new technologies to meet this demand. Massive multiple input multiple output (MIMO) communication systems have great potential for increasing the network capacity of the emerging 5G+ cellular networks. However, leveraging the multiplexing and beamforming gains from these large-scale MIMO systems requires the channel knowlege between each antenna and each user. Obtaining channel information on such a massive scale is not feasible with the current technology available due to the complexity of such large systems. Recent research shows that deep learning methods can lead to interesting gains for massive MIMO systems by mapping the channel information from the uplink frequency band to the channel information for the downlink frequency band as well as between antennas at nearby locations. This thesis presents the research to develop a deep learning based channel mapping proof-of-concept prototype.
Due to deep neural networks' need of large training sets for accurate performance, this thesis outlines the design and implementation of an autonomous channel measurement system to analyze the performance of the proposed deep learning based channel mapping concept. This system obtains channel magnitude measurements from eight antennas autonomously using a mobile robot carrying a transmitter which receives wireless commands from the central computer connected to the static receiver system. The developed autonomous channel measurement system is capable of obtaining accurate and repeatable channel magnitude measurements. It is shown that the proposed deep learning based channel mapping system accurately predicts channel information containing few multi-path effects.
Due to deep neural networks' need of large training sets for accurate performance, this thesis outlines the design and implementation of an autonomous channel measurement system to analyze the performance of the proposed deep learning based channel mapping concept. This system obtains channel magnitude measurements from eight antennas autonomously using a mobile robot carrying a transmitter which receives wireless commands from the central computer connected to the static receiver system. The developed autonomous channel measurement system is capable of obtaining accurate and repeatable channel magnitude measurements. It is shown that the proposed deep learning based channel mapping system accurately predicts channel information containing few multi-path effects.
ContributorsBooth, Jayden Charles (Author) / Spanias, Andreas (Thesis advisor) / Alkhateeb, Ahmed (Thesis advisor) / Ewaisha, Ahmed (Committee member) / Arizona State University (Publisher)
Created2020
Description
Antibiotic resistance is a very important issue that threatens mankind. As bacteria
are becoming resistant to multiple antibiotics, many common antibiotics will soon
become ineective. The ineciency of current methods for diagnostics is an important
cause of antibiotic resistance, since due to their relative slowness, treatment plans
are often based on physician's experience rather than on test results, having a high
chance of being inaccurate or not optimal. This leads to a need of faster, pointof-
care (POC) methods, which can provide results in a few hours. Motivated by
recent advances on computer vision methods, three projects have been developed
for bacteria identication and antibiotic susceptibility tests (AST), with the goal of
speeding up the diagnostics process. The rst two projects focus on obtaining features
from optical microscopy such as bacteria shape and motion patterns to distinguish
active and inactive cells. The results show their potential as novel methods for AST,
being able to obtain results within a window of 30 min to 3 hours, a much faster
time frame than the gold standard approach based on cell culture, which takes at
least half a day to be completed. The last project focus on the identication task,
combining large volume light scattering microscopy (LVM) and deep learning to
distinguish bacteria from urine particles. The developed setup is suitable for pointof-
care applications, as a large volume can be viewed at a time, avoiding the need
for cell culturing or enrichment. This is a signicant gain compared to cell culturing
methods. The accuracy performance of the deep learning system is higher than chance
and outperforms a traditional machine learning system by up to 20%.
are becoming resistant to multiple antibiotics, many common antibiotics will soon
become ineective. The ineciency of current methods for diagnostics is an important
cause of antibiotic resistance, since due to their relative slowness, treatment plans
are often based on physician's experience rather than on test results, having a high
chance of being inaccurate or not optimal. This leads to a need of faster, pointof-
care (POC) methods, which can provide results in a few hours. Motivated by
recent advances on computer vision methods, three projects have been developed
for bacteria identication and antibiotic susceptibility tests (AST), with the goal of
speeding up the diagnostics process. The rst two projects focus on obtaining features
from optical microscopy such as bacteria shape and motion patterns to distinguish
active and inactive cells. The results show their potential as novel methods for AST,
being able to obtain results within a window of 30 min to 3 hours, a much faster
time frame than the gold standard approach based on cell culture, which takes at
least half a day to be completed. The last project focus on the identication task,
combining large volume light scattering microscopy (LVM) and deep learning to
distinguish bacteria from urine particles. The developed setup is suitable for pointof-
care applications, as a large volume can be viewed at a time, avoiding the need
for cell culturing or enrichment. This is a signicant gain compared to cell culturing
methods. The accuracy performance of the deep learning system is higher than chance
and outperforms a traditional machine learning system by up to 20%.
ContributorsIriya, Rafael (Author) / Turaga, Pavan (Thesis advisor) / Wang, Shaopeng (Committee member) / Grys, Thomas (Committee member) / Zhang, Yanchao (Committee member) / Arizona State University (Publisher)
Created2020