Hello everyone! I am Krish, an engineering aspirant and high-school student from Sanskriti School, New Delhi. You can read up more about me on my portfolio page. In a recent interview, the Chief Minister of my state Mr. Arvind Kejriwal said, "The threat of the virus is something that we will have to learn to live with". This statement along with the ongoing lifting of nationwide movement lockdown got me thinking about what happens when the lockdown is completely lifted.
If social media sources are to be believed, the virus took over the NCR with just 15 infected people from Italy, this essentially means that even 15 carriers from a state with 4000+ confirmed cases (as of 15.05.2020) could result in lockdowns all over again and this is a serious threat. While I couldn't figure out a faster, easier and more efficient method for checking for COVID-19, I did come up with a way to check for symptoms of the virus at a domestic level and I am calling it Eyena.
Eyena is an IoT based smart mirror that measures your body temperature upon detecting your presence without any physical contact. Looking like an everyday mirror, it blends with your furniture.
Post the lifting of lockdowns around the world, people will start going places. This will naturally mean a bigger threat of the virus spreading. Especially with the influx and exodus of the crowds such as restaurants, hotels, offices and workplaces, a good idea is to take a thermal reading of the people at the bare minimum before permitting entry.
The problem here however is how time consuming the process is and the peril of the virus spreading due to physical contact. Even in case of IR thermometers, the human proximity is less than what's required for the virus to spread.
Eyena eliminates this problem by ensuring minimal human interaction and labour effort. It also stores each user's data for analysis of trends in each individual user's health. Eyena packs all of this without being tacky and merging with surroundings.
A mockup of our favourite US President using Eyena
Coming back to the seamlessness of Eyena, it can be tweaked to look like any modern day boutique wall mirror
While I believe Eyena is a need in a myriad of spaces, I see the most viable application in restaurants, homes and hotels. Our homes are our safehouses, somewhere we feel protected and we wouldn't want anything breaching our privacy and safety at any costs. Especially with guests who could be potential carriers of the virus, the mirror can check for signs without making anyone feel of being subjected to a test.
A similar procedure can be implemented in hotels and restaurants where guests can get their thermal scan done seamlessly while styling their hair in front of the mirrors.
Due to preventive lockdown measures and my online classes, I haven't been able to give the time Eyena deserves and source all the parts I need for bringing this to reality (and that's somewhere I need your help, I'll discuss this further into this document). Hence, I divided Eyena into 4 progressive versions making it an MVP for the prototyping stage.
While I was going for a purely IR based measurement system earlier (as described in -v1.0 and -v2.0), a major drawback in the MLX96014 is the range in terms of distance. The sensor loses its measurement resolution beyond a distance of more than 10cm which is not viable for a wall mirror. I hence shifted the measurement technology from Infrared to thermal imaging (as described in -v3.0 and -v4.0).
A smart mirror that displays usual stats (weather, news, 5 tweets you should know etc.) from a customizable web API and detects temperature when it detects a face in proximity.
All of -v1.0 + it recognises the face and displays a “Hi + [UserName]” message and stores the data of the user in its database and adds new faces in case of guests. Upon detection, it displays the user’s stats including temperature and analyses the trends and adds it to the web API which is accessible by the owner.
A smart mirror that displays usual stats (weather, news, 5 tweets you should know etc.) from a customizable web API and detects temperature when it detects a face in proximity, it also checks if the user is wearing a mask.
The mask detection of make is done using Maskerizer which is an open source API for face mask detection.
All of -v3.0 + it stores the data of the user in its database and adds new faces in case of guests. Upon detection, it displays the user’s stats including temperature and analyses the trends and adds it to the web API which is accessible by the owner.
This gives rise to the main service I'll be aiming to provide, an ecosystem of IoT devices and a bio ID. The game just went massive "Yeh khel aap se, mujh se, hum sab se bada hai."
The Bio ID is a universal pass for any user that is a guarantee of the person being healthy and safe from the pandemic. The camera in Eyena can read the user's Bio ID QR Code (like the one generated in Aarogya Setu). Upon scanning, Eyena can create (if not already existing) a user in its DB wherein it inserts the current information and measurements of the user (in this case presence of mask and the temperature). If the user accesses more than one Eyenas, the location data can also be added to the analytics. In a way, this acts as a
regulatory entry system, public places can restrict an individual's entry based on a the results from the BioID.
The concept of Bio ID can then be extended to more appliances like Thermal CCTV cameras etc. to create a more diverse ecosystem of connected devices.
Please note that this is the BOM for the prototyping stage, the BOM for the final product would be entirely different and more cost efficient.
To increase the range of measurement in terms of distance from the mirror, we'll be needing thermal imaging instead of IR based temperature measurement. The viable sensor in this case is of the MLX96040 with an output resolution of 32x42 pixels. The calculations for the same are shared below.
General note: Aspect ratio of the image = 4:3
Field of view of the MLX96040 = 55° x 35° [∠EAC]
Desirable distance from the mirror = 100cm (1m) [DC = AB]
Considering right angled △BAC,
tan 27.5° = BC/AB = BC/100
⟹ BC = 100 x ~0.52 = 52cm
⟹ EC = 104cm
∴ Width of coverage at 1m from the mirror = 1.04m
When this image is super imposed onto the 32x24px thermal image grid, horizontally, each thermal pixel covers 3.25cm of the real image at 1m because 1.04m/32 = 3.25cm
The average width of a forehead is 125mm (averaging the curve) and similarly, the average height is 60mm
⟹ The thermally scannable grid will have the dimensions of 2x8 pixels as shown below
My suggestion is that we incorporate 2 camera modules (1 thermal and 1optical) with the same angle of field of view (in the case of MLX96040 55°). The regular optical camera module (with a higher definition) can use face detection APIs to find the position of the face. Upon detection, the thermal image is super imposed on to the regular image to find the thermal location of faces post which the thermal signature is taken.
Consider the following example for a better understanding of how this works -
The regular optical camera detects the position of the face because the resolution of our thermal camera is not high enough
The thermal camera image gets the position of the face through the optical camera for further analysis through superpositioning
Post overlapping the 2 images, we get the following 2x8px grid on the forehead for our measurements -
The extreme pixels of the grid will have a lesser contribution into the calculation of the temperature while the central pixels will give the dominant reading.
Please note that MLX96014 based detection and measurement has been dropped
Eyena is for the community hence, I'd like to build it with the community. So now, I invite everyone interested in bringing this concept to reality with us for collaboration. With the current lockdown situation with minimal access to parts, resources and communication with the world, I believe, coming together and working together would be a great way to gain wider access to resources, ideas and thoughts.
While I welcome everyone who sees Eyena as a learning opportunity onboard, I am particularly looking for people with specifics skill sets to work with to boost the process. I'll be updating this webpage with the developments and you can hit me up anytime for any more details.
Can't find your skillset in the working team but still want to help? You can always be the financial backup, not only can Eyena be a great CSR initiative for your organisation, you also get to be associated with the next potentially big high school startup.
Due to the limitations and cost factors of the current thermal image detection technology, Eyena is not economically viable. However, with the recent developments, we can expect suitable technologies by mid 2021 where I may start working on Eyena (or something similar pertaining to the situation) again.
