ASU Electronic Theses and Dissertations
This collection includes most of the ASU Theses and Dissertations from 2011 to present. ASU Theses and Dissertations are available in downloadable PDF format; however, a small percentage of items are under embargo. Information about the dissertations/theses includes degree information, committee members, an abstract, supporting data or media.
In addition to the electronic theses found in the ASU Digital Repository, ASU Theses and Dissertations can be found in the ASU Library Catalog.
Dissertations and Theses granted by Arizona State University are archived and made available through a joint effort of the ASU Graduate College and the ASU Libraries. For more information or questions about this collection contact or visit the Digital Repository ETD Library Guide or contact the ASU Graduate College at gradformat@asu.edu.
Displaying 1 - 3 of 3
Filtering by
- Creators: Kosut, Oliver
Description
Fundamental limits of fixed-to-variable (F-V) and variable-to-fixed (V-F) length universal source coding at short blocklengths is characterized. For F-V length coding, the Type Size (TS) code has previously been shown to be optimal up to the third-order rate for universal compression of all memoryless sources over finite alphabets. The TS code assigns sequences ordered based on their type class sizes to binary strings ordered lexicographically.
Universal F-V coding problem for the class of first-order stationary, irreducible and aperiodic Markov sources is first considered. Third-order coding rate of the TS code for the Markov class is derived. A converse on the third-order coding rate for the general class of F-V codes is presented which shows the optimality of the TS code for such Markov sources.
This type class approach is then generalized for compression of the parametric sources. A natural scheme is to define two sequences to be in the same type class if and only if they are equiprobable under any model in the parametric class. This natural approach, however, is shown to be suboptimal. A variation of the Type Size code is introduced, where type classes are defined based on neighborhoods of minimal sufficient statistics. Asymptotics of the overflow rate of this variation is derived and a converse result establishes its optimality up to the third-order term. These results are derived for parametric families of i.i.d. sources as well as Markov sources.
Finally, universal V-F length coding of the class of parametric sources is considered in the short blocklengths regime. The proposed dictionary which is used to parse the source output stream, consists of sequences in the boundaries of transition from low to high quantized type complexity, hence the name Type Complexity (TC) code. For large enough dictionary, the $\epsilon$-coding rate of the TC code is derived and a converse result is derived showing its optimality up to the third-order term.
Universal F-V coding problem for the class of first-order stationary, irreducible and aperiodic Markov sources is first considered. Third-order coding rate of the TS code for the Markov class is derived. A converse on the third-order coding rate for the general class of F-V codes is presented which shows the optimality of the TS code for such Markov sources.
This type class approach is then generalized for compression of the parametric sources. A natural scheme is to define two sequences to be in the same type class if and only if they are equiprobable under any model in the parametric class. This natural approach, however, is shown to be suboptimal. A variation of the Type Size code is introduced, where type classes are defined based on neighborhoods of minimal sufficient statistics. Asymptotics of the overflow rate of this variation is derived and a converse result establishes its optimality up to the third-order term. These results are derived for parametric families of i.i.d. sources as well as Markov sources.
Finally, universal V-F length coding of the class of parametric sources is considered in the short blocklengths regime. The proposed dictionary which is used to parse the source output stream, consists of sequences in the boundaries of transition from low to high quantized type complexity, hence the name Type Complexity (TC) code. For large enough dictionary, the $\epsilon$-coding rate of the TC code is derived and a converse result is derived showing its optimality up to the third-order term.
ContributorsIri, Nematollah (Author) / Kosut, Oliver (Thesis advisor) / Bliss, Daniel (Committee member) / Sankar, Lalitha (Committee member) / Zhang, Junshan (Committee member) / Arizona State University (Publisher)
Created2018
Description
The open nature of the wireless communication medium makes it inherently vulnerable to an active attack, wherein a malicious adversary (or jammer) transmits into the medium to disrupt the operation of the legitimate users. Therefore, developing techniques to manage the presence of a jammer and to characterize the effect of an attacker on the fundamental limits of wireless communication networks is important. This dissertation studies various Gaussian communication networks in the presence of such an adversarial jammer.
First of all, a standard Gaussian channel is considered in the presence of a jammer, known as a Gaussian arbitrarily-varying channel, but with list-decoding at the receiver. The receiver decodes a list of messages, instead of only one message, with the goal of the correct message being an element of the list. The capacity is characterized, and it is shown that under some transmitter's power constraints the adversary is able to suspend the communication between the legitimate users and make the capacity zero.
Next, generalized packing lemmas are introduced for Gaussian adversarial channels to achieve the capacity bounds for three Gaussian multi-user channels in the presence of adversarial jammers. Inner and outer bounds on the capacity regions of Gaussian multiple-access channels, Gaussian broadcast channels, and Gaussian interference channels are derived in the presence of malicious jammers. For the Gaussian multiple-access channels with jammer, the capacity bounds coincide. In this dissertation, the adversaries can send any arbitrary signals to the channel while none of the transmitter and the receiver knows the adversarial signals' distribution.
Finally, the capacity of the standard point-to-point Gaussian fading channel in the presence of one jammer is investigated under multiple scenarios of channel state information availability, which is the knowledge of exact fading coefficients. The channel state information is always partially or fully known at the receiver to decode the message while the transmitter or the adversary may or may not have access to this information. Here, the adversary model is the same as the previous cases with no knowledge about the user's transmitted signal except possibly the knowledge of the fading path.
First of all, a standard Gaussian channel is considered in the presence of a jammer, known as a Gaussian arbitrarily-varying channel, but with list-decoding at the receiver. The receiver decodes a list of messages, instead of only one message, with the goal of the correct message being an element of the list. The capacity is characterized, and it is shown that under some transmitter's power constraints the adversary is able to suspend the communication between the legitimate users and make the capacity zero.
Next, generalized packing lemmas are introduced for Gaussian adversarial channels to achieve the capacity bounds for three Gaussian multi-user channels in the presence of adversarial jammers. Inner and outer bounds on the capacity regions of Gaussian multiple-access channels, Gaussian broadcast channels, and Gaussian interference channels are derived in the presence of malicious jammers. For the Gaussian multiple-access channels with jammer, the capacity bounds coincide. In this dissertation, the adversaries can send any arbitrary signals to the channel while none of the transmitter and the receiver knows the adversarial signals' distribution.
Finally, the capacity of the standard point-to-point Gaussian fading channel in the presence of one jammer is investigated under multiple scenarios of channel state information availability, which is the knowledge of exact fading coefficients. The channel state information is always partially or fully known at the receiver to decode the message while the transmitter or the adversary may or may not have access to this information. Here, the adversary model is the same as the previous cases with no knowledge about the user's transmitted signal except possibly the knowledge of the fading path.
ContributorsHosseinigoki, Fatemeh (Author) / Kosut, Oliver (Thesis advisor) / Zhang, Junshan (Committee member) / Sankar, Lalitha (Committee member) / Bliss, Daniel (Committee member) / Arizona State University (Publisher)
Created2019
Description
Modern digital applications have significantly increased the leakage of private and sensitive personal data. While worst-case measures of leakage such as Differential Privacy (DP) provide the strongest guarantees, when utility matters, average-case information-theoretic measures can be more relevant. However, most such information-theoretic measures do not have clear operational meanings. This dissertation addresses this challenge.
This work introduces a tunable leakage measure called maximal $\alpha$-leakage which quantifies the maximal gain of an adversary in inferring any function of a data set. The inferential capability of the adversary is modeled by a class of loss functions, namely, $\alpha$-loss. The choice of $\alpha$ determines specific adversarial actions ranging from refining a belief for $\alpha =1$ to guessing the best posterior for $\alpha = \infty$, and for the two specific values maximal $\alpha$-leakage simplifies to mutual information and maximal leakage, respectively. Maximal $\alpha$-leakage is proved to have a composition property and be robust to side information.
There is a fundamental disjoint between theoretical measures of information leakages and their applications in practice. This issue is addressed in the second part of this dissertation by proposing a data-driven framework for learning Censored and Fair Universal Representations (CFUR) of data. This framework is formulated as a constrained minimax optimization of the expected $\alpha$-loss where the constraint ensures a measure of the usefulness of the representation. The performance of the CFUR framework with $\alpha=1$ is evaluated on publicly accessible data sets; it is shown that multiple sensitive features can be effectively censored to achieve group fairness via demographic parity while ensuring accuracy for several \textit{a priori} unknown downstream tasks.
Finally, focusing on worst-case measures, novel information-theoretic tools are used to refine the existing relationship between two such measures, $(\epsilon,\delta)$-DP and R\'enyi-DP. Applying these tools to the moments accountant framework, one can track the privacy guarantee achieved by adding Gaussian noise to Stochastic Gradient Descent (SGD) algorithms. Relative to state-of-the-art, for the same privacy budget, this method allows about 100 more SGD rounds for training deep learning models.
This work introduces a tunable leakage measure called maximal $\alpha$-leakage which quantifies the maximal gain of an adversary in inferring any function of a data set. The inferential capability of the adversary is modeled by a class of loss functions, namely, $\alpha$-loss. The choice of $\alpha$ determines specific adversarial actions ranging from refining a belief for $\alpha =1$ to guessing the best posterior for $\alpha = \infty$, and for the two specific values maximal $\alpha$-leakage simplifies to mutual information and maximal leakage, respectively. Maximal $\alpha$-leakage is proved to have a composition property and be robust to side information.
There is a fundamental disjoint between theoretical measures of information leakages and their applications in practice. This issue is addressed in the second part of this dissertation by proposing a data-driven framework for learning Censored and Fair Universal Representations (CFUR) of data. This framework is formulated as a constrained minimax optimization of the expected $\alpha$-loss where the constraint ensures a measure of the usefulness of the representation. The performance of the CFUR framework with $\alpha=1$ is evaluated on publicly accessible data sets; it is shown that multiple sensitive features can be effectively censored to achieve group fairness via demographic parity while ensuring accuracy for several \textit{a priori} unknown downstream tasks.
Finally, focusing on worst-case measures, novel information-theoretic tools are used to refine the existing relationship between two such measures, $(\epsilon,\delta)$-DP and R\'enyi-DP. Applying these tools to the moments accountant framework, one can track the privacy guarantee achieved by adding Gaussian noise to Stochastic Gradient Descent (SGD) algorithms. Relative to state-of-the-art, for the same privacy budget, this method allows about 100 more SGD rounds for training deep learning models.
ContributorsLiao, Jiachun (Author) / Sankar, Lalitha (Thesis advisor) / Kosut, Oliver (Committee member) / Zhang, Junshan (Committee member) / Dasarathy, Gautam (Committee member) / Arizona State University (Publisher)
Created2020