Minimum Entropy Coupling with Bottleneck
This paper investigates a novel lossy compression framework operating under
logarithmic loss, designed to handle situations where the reconstruction
distribution diverges from the source distribution. This framework is
especially relevant for applications that require joint compression and
retrieval, and in scenarios involving distributional shifts due to processing.
We show that the proposed formulation extends the classical minimum entropy
coupling framework by integrating a bottleneck, allowing for a controlled
degree of stochasticity in the coupling. We explore the decomposition of the
Minimum Entropy Coupling with Bottleneck (MEC-B) into two distinct optimization
problems: Entropy-Bounded Information Maximization (EBIM) for the encoder, and
Minimum Entropy Coupling (MEC) for the decoder. Through extensive analysis, we
provide a greedy algorithm for EBIM with guaranteed performance, and
characterize the optimal solution near functional mappings, yielding
significant theoretical insights into the structural complexity of this
problem. Furthermore, we illustrate the practical application of MEC-B through
experiments in Markov Coding Games (MCGs) under rate limits. These games
simulate a communication scenario within a Markov Decision Process, where an
agent must transmit a compressed message from a sender to a receiver through
its actions. Our experiments highlight the trade-offs between MDP rewards and
receiver accuracy across various compression rates, showcasing the efficacy of
our method compared to conventional compression baseline.
6725296eccb533a2ed462bfa
M. Reza Ebrahimi
claimed_verified
2024-11-03T14:20:51.667Z
648a1075e8bee5332919e0ab
https://cdn-avatars.huggingface.co/v1/production/uploads/648a1075e8bee5332919e0ab/KDzi904zGPuy_PL3gbUYE.jpeg
M.Reza Ebrahimi
user
mamaj92
6725296eccb533a2ed462bfb
Jun Chen
6725296eccb533a2ed462bfc
Ashish Khisti
Minimum Entropy Coupling with Bottleneck
This paper investigates a novel lossy compression framework operating under
logarithmic loss, designed to handle situations where the reconstruction
distribution diverges from the source distribution. This framework is
especially relevant for applications that require joint compression and
retrieval, and in scenarios involving distributional shifts due to processing.
We show that the proposed formulation extends the classical minimum entropy
coupling framework by integrating a bottleneck, allowing for a controlled
degree of stochasticity in the coupling. We explore the decomposition of the
Minimum Entropy Coupling with Bottleneck (MEC-B) into two distinct optimization
problems: Entropy-Bounded Information Maximization (EBIM) for the encoder, and
Minimum Entropy Coupling (MEC) for the decoder. Through extensive analysis, we
provide a greedy algorithm for EBIM with guaranteed performance, and
characterize the optimal solution near functional mappings, yielding
significant theoretical insights into the structural complexity of this
problem. Furthermore, we illustrate the practical application of MEC-B through
experiments in Markov Coding Games (MCGs) under rate limits. These games
simulate a communication scenario within a Markov Decision Process, where an
agent must transmit a compressed message from a sender to a receiver through
its actions. Our experiments highlight the trade-offs between MDP rewards and
receiver accuracy across various compression rates, showcasing the efficacy of
our method compared to conventional compression baseline.
A novel lossy compression framework with a bottleneck extension to minimum entropy coupling is proposed, providing theoretical insights and practical applications in Markov Coding Games.
logarithmic loss minimum entropy coupling bottleneck Entropy-Bounded Information Maximization Minimum Entropy Coupling Markov Coding Games