Our presentation relying on JATD focuses on the semiosis issue, i.e. the way signs are built and deciphered in the teaching-learning process. Following Sensevy (2011), we will show how didactic transactions are “grounded in the reciprocal semiosis of others" (p. 68). Indeed we assume that both, teacher and students, although in different institutional positions, have to decipher two types of signs: 1) signs of the didactic milieu, and 2) signs of the didactic contract. Two empirical analyses will exemplify our inquiry. The first analysis (Gruson) will refer to video conferencing sessions during which primary French and English students interact. For this presentation, we will show that the joint action relies on a specific equilibrium between the didactic contract and the milieu embodied by a two-fold semiosis. The second analysis (Loquet) refers to gymnastics teaching sequences, in physical education. We analyse how the teacher and students, during a sequence based on the use of a material structure called "gym kit", are able to exploit the milieu to build skills that approximate the experts’ gymnastic skills. We argue that the double semiosis process is based on the ability to perceive the same type of information from objects present in the environment.