A Theoretically Novel Trade-off for Sparse Secret-key Generation

by   Makan Zamanipour, et al.

We in this paper theoretically go over a rate-distortion based sparse dictionary learning problem. We show that the Degrees-of-Freedom (DoF) interested to be calculated - satnding for the minimal set that guarantees our rate-distortion trade-off - are basically accessible through a Langevin equation. We indeed explore that the relative time evolution of DoF, i.e., the transition jumps is the essential issue for a relaxation over the relative optimisation problem. We subsequently prove the aforementioned relaxation through the Graphon principle w.r.t. a stochastic Chordal Schramm-Loewner evolution etc via a minimisation over a distortion between the relative realisation times of two given graphs 𝒢_1 and 𝒢_2 as 𝕄in_𝒢_1, 𝒢_2 𝒟( t( 𝒢_1 , 𝒢) , t( 𝒢_2 , 𝒢) ). We also extend our scenario to the eavesdropping case. We finally prove the efficiency of our proposed scheme via simulations.



There are no comments yet.


page 2


Computational Intractability of Dictionary Learning for Sparse Representation

In this paper we consider the dictionary learning problem for sparse rep...

Online Orthogonal Dictionary Learning Based on Frank-Wolfe Method

Dictionary learning is a widely used unsupervised learning method in sig...

Lossy Compression via Sparse Linear Regression: Computationally Efficient Encoding and Decoding

We propose computationally efficient encoders and decoders for lossy com...

Distortion Varieties

The distortion varieties of a given projective variety are parametrized ...

On the exact recovery of sparse signals via conic relaxations

In this note we compare two recently proposed semidefinite relaxations f...

Multi Terminal Probabilistic Compressed Sensing

In this paper, the `Approximate Message Passing' (AMP) algorithm, initia...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Many machine learning and modern signal processing applications

such as bio-metric authentication/identification and recommending systems , follow sparse signal processing techniques [1, 2, 3, 4, 5, 6]

. The sparse synthesis model focuses on those data sets that can be approximated using a linear combination of only a small number of cells of a dictionary. The applications of sparse coding based dictionary learning go chiefly over either or both extraction and estimation of local features. Typically, this kind of discipline is controlled via a prior decomposition of the original signal into overlapping blocks

call side-information. In relation to this strategy, there mainly exist two demerits: (i) each point in the signal is estimated multiple times something that shows a redundancy; and (ii) a shifted version of the features interested to be learnt are captured since the correlations/side-information among neighboring data sets are not fully taken into account something that results in the inevitable fact that some portions of the learnt dictionaries may be practically of a partially useless nature.

A quick overview over the literature in terms of key-generation and authentication is strongly widely provided here in details. In [7] and for cloud-assisted autonomous vehicles, Q. Jiang et al technically proposed a bio-metric privacy preserving three-factor authentication and key agreement. In [8] and through vehicular crowd-sourcing, F. Song et al technically proposed a privacy-preserving task matching with threshold similarity search. In [9] and for mobile Internet-of-Things (IoT), X. Zeng, et al technically proposed a highly effective anonymous user authentication protocol. In [10] and for fog-assisted IoT, J. Zhang et al technically proposed a revocable and privacy-preserving decentralized data sharing scheme. In [11] and for resource-constrained IoT devices, Q. Hu et al technically proposed a half-duplex mode based secure key generation method. In [12] and for mobile crowd-sensing under un-trusted platform, Z. Wang et al theoretically explored privacy-driven truthful incentives. In [13] and for IoT-enabled probabilistic smart contracts, N. S. Patel et al theoretically explored blockchain-envisioned trusted random estimators. In [14] and for IoT, R. Chaudhary et al theoretically explored Lattice-based public key crypto-mechanisms. In [15] and for vehicle to vehicle communication, V. Hassija et al

theoretically explored a scheme with the aid of directed acyclic graph and game theory. In

[16] and for vehicular social networks, J. Sun et al theoretically explored a secure flexible and tampering-resistant data sharing mechanism. In [17] and for mission-critical IoT applications, H. Wang et al theoretically explored a secure short-packet communications. In [18] and for edge-assisted IoT, P. Gope et al theoretically proposed a highly effective privacy-preserving authenticated key agreement framework.

Overall, many works have been fulfilled in the context of secret key-generation, however, sparse coding and an optimised problem in this area has not been introduced so far. In addition, although the literature has tried its best to propose novel frameworks [7, 9, 10, 13, 16, 17, 18] as well as the optimisation techniques [8, 11, 12, 14, 15] and the relative relaxations, the population of the research and work done in this context still essentially lacks and requires to be further enhanced.

Fig. 1: Essential details of our problem-and solution and its flow.

I-a Motivations and contributions

In this paper, we are interested in responding to the following question: How can we guarantee highly adequate relaxations over a rate-distortion based sparse dictionary learning? How is Alice able to efficiently utilise totally disparate resources to guarantee her private message to be concealed from Eve? With regard to the non-complete version of the literature, the expressed question strongly motivate us to find an interesting solution, according to which our contributions are fundamentally described as follows.

  • (i) We initially write a rate-distortion based sparse coding optimisation problem. Subsequently, we try to add some extra constraints which are of a purely meaningful nature in this context.

  • (ii) We show that we can get access to the time evolution of the Degrees-of-Freedom (DoF) in which we are interested. We do this through a Langevin equation.

  • (iii) We then add an additional constraint in relation to the side-information.

  • (iv) Pen-ultimately, we make trials to theoretically relax the resultant optimisation problem w.r.t. the facts of Graphon and a stochastic Chordal Schramm-Loewner evolution etc. something that can be chiefly elaborated according to some theoretical principles such as Riemannian gradients in Riemannian geometry and the Kroner’s graph entropy.

  • (v) Finally, we also extend our scenario to the eavesdropping case. We in fact technically define some probabilistic constraints to be added to the main optimisation problem.

I-B General notation

The notations widely used throughout the paper is given in Table I.

Notation Definition Notation Definition
Problem Problem
Problem New Version of Problem
Perturbation Expected-value
Time Instance Side-information
Distributed Random-walk
Specific Time Instance Arbitary Threshold
Distortion Function Mutual Information
Graph Brownian Motion
TABLE I: List of notations.

I-C Organisation

The rest of the paper is organised as follows. The system set-up and our main results are given in Sections II and III. Subsequently, the evaluation of the framework and conclusions are given in Sections IV and V.

Ii System model and problem formulation

In this section, we describe the system model, subsequently, we formulate the basis of our problem.

Ii-a Flow of the problem-and-solution

Initially speaking, take a quick look at Fig. 1. It basically says:

  • Problem-and-Solution is to find the minimal set which guarantees a rate-distortion trade-off;

  • DoF stands fundamentally theoretically for the minimal set expressed above which essentially talk about the transition jumps;

  • Trade-off is the rate-distortion problem;

  • Reason of side-information and the logic behind of using that is to control the sparse coding111Something that is traditional as mentioned in the paper in the next parts.;

  • Sparse secret-key generation and the logic behind of using that is to guarantee a correlated minimal set available between Bob and Eve;

  • Time evolution of DoF is explored via a Langevin equation;

  • Relaxation of the rate-distortion problem is interpreted through the Graphon principle and w.r.t. a stochastic Chordal Schramm-Loewner evolution; and

  • Logic of time in optimisation is to guarantee an instantaneous optimisation.

Ii-B System description

A traditional sparse coding scheme practically includes the following steps [1]:

  • Sender: With the aid of a sparsifying transform which should be trained to the receiver call Bob via the server, the sender call Alice generates the sparse code-words from her own data. She subsequently shares the privacy-protected sparse code-book with the the server as the service provider. Alice now decides222According to the Kerckchoffs’s principle in cryptography [1]. to permit a licensed availability, i.e., publicity over the sparsifying transform learnt by the client(s).

  • Server Step 1 as an indexing for the sender: The server marks some indexes on the received sparse-codes in a database.

  • Client: Via the shared sparsifying transform, Bob as the client generates a sparse representation from his query data and subsequently sends it to the server.

  • Server Step 2 as a matching test for the client: The server searches to find the most similar sparse-code set, and subsequently replies to the client’s request.

In fact, Alice has a secret message while she decides to send it to Bob in the presence of the passive potential or/and active actual Eve(s).

Ii-C Main problem

A sparse dictionary learning problem can be mathematically formulated as [1, 2, 3, 4, 5, 6]


with regard to the thresholds and , where denotes the observed data, represents the unknown dictionary, stands for the original data set w.r.t. the data instance , refers to the sparse representation coefficient set, is a distribution over appropriate perturbations [6], while we have the pre-trained models and .

Definition 1333See e.g. [19, 20].: secret key.

A random variable

with the finite range information-theoretically represents an secret key for Alice and Bob, achievable through the public communication , if there exist two functions and such that for any the following three conditions hold: (i) ; (ii) ; and (iii)

. These three conditions respectively ensure that (i) Alice and Bob generate the same secret key with high probability as much as possible; the generated secret key is efficiently kept hide from any user except for Bob who gets access to the public communication

; and (iii) the secret key is sub-uniformly distributed.

This definition is based fully upon a passive Eve which can be extended to an active scenario although we in the next parts, use the active one.

Iii Main results

In this section, our main results are theoretically provided in details.

Iii-a Time evolution of the sparsity

Proposition 1

The time evolution of the sparsity values, that is, as DoF relating to the transition jumps for the rate-distortion based sparse coding optimisation problem expressed in Eqn. (1) can be mathematically attainable for the distortion term while for the instant .

Proof: See Appendix A.

Iii-B Relaxation of the rate-distorion problem

Proposition 2

While the degrees of freedom of interest so far is , following the previous result, i.e., Proposition 1 and its relative proof given in Appendix A, the main problem over the distortion term is hard. However, this problem can be theoretically relaxable through a new definition over the given side-information as a new distortion constraint .

Proof: See Appendix B.

Algorithm 1 can solve given in Eqn. (6) in Appendix B.

  Initialisation. while do (i) Solve given in Eqn. (6); and (ii) Update. endwhile Output. end
Algorithm 1 A greedy algorithm to the Problem .

Iii-C Positive definity

Proposition 3

DoF, if they are non-zero, they are conditionally positive definite.

Proof: See Appendix C.

Iii-D Extension to the eavesdropping scenraio

Assume444See e.g. [21, 22]. that we have two users, without loss of generality one of which is Bob as the legitimate user, and another one is an eavesdropper call Eve who can get access to the information in relation to the private message arose between Alice and Bob.

Proposition 4

For the th instant, call the sparsity patterns and respectively for Bob and Eve, that is, in relation to the channel between Alice and Bob, and the one between Alice and Eve, respectively. For the th instant and th sub-channel, define555See e.g. [21, 22]. . The Problem expressed in Eqn. (6) can be re-written as the Problem expressed in Eqn. (7), or the Problem expressed in Eqn. (8) or even the Problem expressed in Eqn. (9).

Proof: See Appendix D.

Iv Numerical results

We have done our simulations w.r.t. the Bernoulli-distributed data-sets using GNU Octave of version

on Ubuntu .

Fig. 2 shows the optimum value earned from the Problem given in Eqn. (6) while and are divided by the one when , i.e., the case in which we do not constrain our optimisation problem w.r.t. the relative constraints. As totally, obvious, the more constraints we intelligently and purposefully provide, the significantly more favourable response we can undoubtedly guarantee with regard to our proposed scheme.

Fig. 3 shows the instantaneous key-rate earned from the Problem given in Eqn. (9) while and are divided by the one when , i.e., the case in which we do not constrain our optimisation problem w.r.t. the relative constraints. The same as the previous trial, the more constraints we smartly provide, the considerably more adequate response we consequently assure in relation to our proposed scheme.

Fig. 2: Instantaneous optimum value vs. iterations: How Algorithm 1 converges w.r.t. the thresholds and in the Problem given in Eqn. (6).
Fig. 3: Instantaneous key-rate vs. iterations: How Algorithm 1 converges w.r.t. the thresholds and in the Problem given in Eqn. (9).

V conclusion

We have theoretically shown a novel solution to an hard rate-distortion based sparse dictionary learning in the context of a novel method of relaxation. We theoretically showed the accessibility of DoF in which er are interested via a Langevin equation. In fact, the relative time evolution of DoF, i.e., the transition jumps for a relaxation over the relative optimisation problem was calculated in the context of a tractable fashion. The aforementioned relaxation was proven with the aid of the Graphon principle w.r.t. a stochastic Chordal Schramm-Loewner evolution etc. Our proposed method and solutions were extended to the eavesdropping case. The effectiveness of our proposed framework was ultimately shown by simulations.

Appendix A Proof of Proposition 1

The problem written in Eqn. (1) can be theoretically re-expressed as


while for which holds, or correspondingly in the context of an instantaneous666See e.g. [23, 24] to understand what an instantaneous expected-value it means. optimisation problem


Now, as we are interested in theoretically exploring the sparsity, as DoF in relation to the transition jumps, one can apply the Langevin equation according to which the time evolution of the aforementioned DoF can be accessible. Thus, we have


while is an arbitrary parameter, and the minus sign principally shows a backward orientation in the solution to the problem, as well as the theoretical fact that is a random-walk.

The proof is now completed.

Appendix B Proof of Proposition 2

The proof is provided here in terms of the following multi-step solution.

Step 1. It is strongly conventional [25, 26] to define a constraint over a distortion between the side-information and the sparse data set. Therefore, we do this here as well according to which our optimisation problem is conveniently re-written as


w.r.t. the arbitary threshold .

Step 2. One can here say that DoF cited above can be in terms of the possibly available ways via which one can calculate a specific amount of information. Accordingly, the possibly available pieces of a specific amount of information calculable such as the relative side-information set can be DoF intersted here to be calculated.

Step 3. Now, assume that the possibly available pieces of the specific amount of information calculable talked above are in the context of probabilistic energy777In order to see the relevance between energy and information see e.g. [27, 28, 29]. quanta. Subsequently, we assume a given Ball in which the number of energy quanta should require

while is in relation to , and we have a probability density function (pdf) while , and stand for the energy, momentum and time, respectively. Subsequently, we assign

while is the second derivative in the polar axis. Thus, defining

the relative energy quanta numbers is calculated as


Step 4. Now, theoretically imagine a game and the relative users as follows. The Reward is each of the possibly available pieces of the specific amount of information calculable in the previous step, for each relative DoF as the users.

Remark 1. In our game, the population physically has a positive growth rate. This is because of the fact that the principle of Diffusion-and-Reaction hold here, that is, a diffusive flow exists from the side of a higher concentration to a less one, and reciprocally, a reactive movement exists from the less concentration towards the higher one.

Step 5. Now, let us mathematically define an initial condition for the Langevin based constraint. We have . Now, call the Picard-Lindelof Theorem888See e.g. [30] to see what it is. which can be of a purely useful nature to say that: the Langevin equation

w.r.t. the initial condition can have a unique solution w.r.t. a surely existing over the zone ; while is uniformly Lipschitz continuous in and continuous in . Thus, we indeed branch and localise the optimisation problem into its localities, without loss of generality and optimality. This is because of the hard structure of our main optimisation problem since that is not convex, so, the strong duality does not theoretically hold [31].

Step 6. Re-call Remark 1 for the fact that the population size of the relative users has a growth over the time zone, according to which we can theoretically consider a Graphon999See e.g. [6]. in between. Indeed, we can prove that there exists a duality between the main optimisation problem and its localised version created in Step 5. Now, w.r.t. the fact that there would exist a stochastic Chordal101010Which maps a boundary point to another one, which is in contrast to the Radial Schramm-Loewner evolution which maps a boundary point to an interior one: See e.g. [32, 33] to see what they are. Schramm-Loewner evolution111111See e.g. [32, 33] to see what it is. which shows a Brownian motion over the surface of the relative evolving Riemannian-manifold relating to the graph as

w.r.t. the arbitary function and the random-walk . On other hand, since a graphon as a density measure is the number of times occurs as a sub-graph of the graph , and w.r.t. the fact that two graphs, do they have the final approximation in terms of a same graphon, they actualise the same phenomenon [34], and regarding the fact that the positive growth of the population size literally defined above highly requires an adjacent matrix for the evolving Riemannian-manifold and the graph in between, a proof for a the duality we are looking for can be proven. Now, the duality basically holds iff and only iff and conclude a common phenomenon, while fundamentally is a stochastic Schramm-Loewner evolutioned version of in the nearest physical vicinity of the Reimannian sub-manifold, relating to the surface of the relative evolving Riemannian-manifold, that is

while is a sufficiently small non-zero value and and respectively show the final realisation of the relative graphs and . This is equivalent to121212This kind of optimisation problem can be solved by the alternating direction method of multipliers: See e.g. [27, 28, 29].

Towards such end, we re-express our optimisation problem as Eqn. (6) on the top of the next page in which the term indicates the stochastic Schramm-Loewner evolution expressed above and is an arbitary threshold to constraint the amount of the Schramm-Loewner evolution. Now, let us continue by the following definition.

Definition 2131313See e.g. [35].: The order binomial extension. Consider the mapping such that holds and a

which is a discrete random variable that takes values on the non-zero non-negative integers

. The mapping such that holds is theoretically called the order binomial extension of if .

In the subsequence of Definition 2, one can re-consider the optimisation


while basically is the probability mass function (pmf) of . It is also importantly interesting to be noticed that the term comes principally from the convolutional nature of the Graph associated fundamentally with since the terms and may be technically of a Poisson-randomly-distributed nature. Meanwhile, we may interchangeably use with a slight abuse of notation here, without loss of generality.

Now, consider the graph and the subgraphs and with respectively random vertex sets , and . Let , and be the probability density distributions respectively on , and whose marginal on are equal to . Now the later optimisation can be re-written as

where holds, w.r.t. the non-zero non-negative threshold as well as arbitary functions and , while the first two additional constraints are thoroughly justified according to the Korner’s graph entropy141414[36, 37]. something that can be stongly followed up via Riemannian gradients in Riemannian geometry as described below. Indeed, the first two-constraints have come from the fact that is equivalent to

The strongly theoretical logic behind of our interpretation and consideration over the evolution expressed above can be physically justified as follows as well, according to the Riemannian geometry and Riemannian optimisation principles151515 See e.g. [38, 39, 40].. Let us initially define the Riemannian gradients161616 See e.g. [38, 39, 40, 41, 42]. in relation to the sub-manifolds as the orthogonal projection of the distortion function


onto the associated tangent space as

which can be widely re-written in terms of the Riemannian logarithmic map operator

while and theoretically are the eigen-value matrices associated with the graphs and , respectively. Now, defining translation functions and respectively for the real-parts of and , and rotation functions and for the relative imaginary-parts, while the following conditions satisify

while principally indicates the average value and and are Brownian motions. Now, our relaxation we are focusing on is equivalent ot the following optimisation

Lemma 1

For the equation derived above, the necessary and sufficient condition to be relaxable fundamentally is the fact that there should exist and such that the following satisfies

Proof: The proof is omitted for brevity.

Step 7. Finally, the following lemma can help us in concluding our interpretation.

Lemma 2

The constraint

is interpreted according to the Bochner’s theorem.

Proof: The Bochner’s theorem171717see e.g. [43] to see what it is. says that for a locally compact abelian group181818A group while each pair of group members are interchangible. , with dual group , there exists a unique probability measure such that

holds. This completes the proof of Lemma 1 here.

The proof is now completed.

Appendix C Proof of Proposition 3

The Herglotz-Riesz representation theorem for harmonic functions191919see e.g. [44] to see what it is. can be useful here for us saying that DoF we are interested in, if they are non-zero, have positive real-parts iff and only iff there is a probability measure on the unit circle such that

holds, while

since it is proven that is harmonic, that is

according to the Newton’s second law of motion, roughly speaking.

The minimisation defined above can be re-written as

for the case where is holomorphic.

Remark 3. The term can be re-expressed as

while . In this term, the term shows the story in relation to the tangent space expressed in Step 6 in Appendix B.

The proof is now completed.

Appendix D Proof of Proposition 4

The achievable secret-key rate per dimension is202020see e.g. [21, 22].

while holds, and is the instantaneous signal-to-noise (SNR), as well as the fact that


hold212121see e.g. [21, 22]., while is the observed data by Eve.

Now, one can assert that the Problem expressed in Eqn. (6) can be re-written as the Problem expressed in Eqn. (7) or the Problem expressed in Eqn. (8), while for the th instant and th sub-channel, we have

and we principally define the following probabilistic constraint

which can be in parallel with

w.r.t. the arbitrary thresholds , , and , roughly speaking.

The two probabilistic constraints theoretically introduced here stand respectively for the following facts: (i) the upper-bound of the probability value should be constrained through a threshold; (ii) and the lower-bound of the instantaneous SNR should also be constrained through a threshold as well.

Instead of the two constraints given above, one can constrain the secret-key rate as

w.r.t. the arbitrary thresholds and .

Remark 4. It is absolutely interesting to note that although perturbations are considered in the literature mainly due to attacks, we do not ignore the term in Eqns. (7-8-9) in order to preserve the optimality and generality of the problem.

Remark 5. The three probabilistic constraints newly declared above are relaxable according to the principle of the Concentration-of-Measure something that theoretically proves an exponential bound for the two aforementioned constraints222222See e.g. [27, 28, 29] for further discussion..

The proof is now completed.