Marker-based image tracking using MindAR and React
MindAR is a web-based augmented reality library that integrates with Three.js and A-frame. This library supports both marker-based AR and face-tracking. In this Answer, we’ll focus on creating a React application using Three.js with MindAR. Currently, incorporating augmented reality into websites, particularly e-commerce sites, is popular. There are many libraries and frameworks available to assist with this task.
Project goal
In this demo, we want a 3D educative logo to overlay over an image that has been placed inside the scope of the camera.
We can download the “target” image from below and compile the image to be used in the application.
Alternatively, we can simple download the “.mind” file below.
The overlay image could be a 3D asset or any image. In our case we’ll be overlaying educative logo over the target image. We can download the 3D logo asset from below.
Setting up a React project
We can use the commands given below to set up the project quickly.
npx create-react-app <Project_Name>
cd <Project_Name>
Setting up the packages
Now, it’s time to add the necessary packages (mind-ar and three) to our application.
npm i mind-ar --save
npm install --save three
Project structure
We’ll create a few more folders for better project structuring. The final project structure would look like the following:
The
Componentsfolder contains the component related to the MindAR.The
threeDAssetsfolder contains the three assets, like the target and overlay models.
In the code sample below, we have used “.glb” models and “.mind” links instead of the components folder. While it’s preferable to upload the models to a CDN and fetch them from there, in small applications, we can create a folder like “threeDAsset.” We can then use the relative path within the application.
Code sample
Press the “Run” button below and click on the link.
ã F ) 9 5 @@ ° n PNG
IHDR (-S äPLTE""""""""""""""""""2PX=r)7;*:>H¤-BGE8do5Xb6[eK®K¯1MU9gs3S\I§:gt'03@{V¹ÔT´ÏA}V»Ö@y6\fH¦-CII¨E+;@7_i7_jFJ«K°H£-BHaÚû,@FCL³&.0W½ÙN£ºI¨$)+BJªR¯È?v>s>uS±Ê=qP©ÁP¨ÀP§¿,?D4U^%+-M ¶K®%+,2OX+<AL²#&&D%,.I©vôTö tRNSIæçJäeÀe¦ ©IDATxMµZEA ÿÙ³ îî%R¡ïßáTThÇG
»,Á®Å=²ÒîmífímnfA$â>!¦gºôHg½Eßܵ} Ý»ýº¼kdú§¯JoÎ3æL"J¹ ÌÕüQ$âçļ ffµ,é5i9̯H¨ /mBwÍÜw;D
Ø+&W«ª¹¨Dôo@Ê´RI©ÐB¡om.Û³À IEND®B`PNG
IHDR שÍÊ ePLTE""""""""""""""""""""""""2RZN¢¹J«3R[J¬)59YÁÞ0KS4W`Q«ÄL²%+-0JR)6::gtC"##?vU·Ñ?w<n{&-/YÂß=q:iuBA}A{B/IPP§¿=qK®_ÔóL³$();lzR¯ÉaÚûI¨ZÆã3U^1MU3T]ZÅâI§X¿ÜF-BGP¨À6[e,@E5ZdO§¿-BHX¿Û+=AW¾Ú,@FW¼ØQªÃ?v
W¼×+<A@y"#$\Ìê4Wa\ÌëS²Ì$(*.EL^ÑñVºÕ6]h#$%G¡#&';jwV¹Ô-CIL±ZÄá^Ðï>uS°Ê/HNM ·_Õõ\ËéM ¶8doD
D>t+=B[Èæ,>C>t<o}@y0LS.EKT´Î$'(%,.A~W½ÙC%+,\ÊèC!ä tRNSíîG¾Ö OIDATxlÃB¶Q
u´ß_ȳ<˦Ýv eê²óa6Aξûv¢{@Î E' Þd IÕ!ç í ðCÔT þg 1ÂE(ÏñSQsâi
Ä
Zÿ·V¹ Ð)ég!ªhÎùtéº-i}µµ<Õ?¶lBZaÄ´4{DÓâ»_e8¥yÇÀ 3)¥?°f;8.ã ¤tÌ=å; :ã52fKZìlù¨ØÍ9.#ÒAÁqÌúÛ ®£Vÿ`=$¬Â?_¶¾®ÔqMç.ïJ$
?^q÷ñíÛï.},ìsæÝ _TttÔ¾ 1#/(ì-[è`è`ÌÚïÅðZ d5?ÎebZ¿Þi.Ûæ ìqÎ+1°}Â5ù ïçd¨GÏø IEND®B`PNG
IHDR D¤Æ APLTE """""""""""""""""""""""""""2RZVºÖ_ÔôU·Ñ=r$()'25]ÎíC0LS<o}XÀÜX¿Û0JQ=p~D<n{VºÕE8do_ÔóEFH¥9dp_ÕõH¤I¨F6[e`Ö÷`×øL³/GM_ÓòU¸Ó'02P©Á/IPPªÂX¿Ü&/1;ly3R[`ØøG¡T³Í\ÌêaÚû1OW"##Q«ÄaÙúR®Ç=q`Öö.EL+=ATµÐ-CIK®#&'C^ÐïI¨&.04U^^Ñð@yZÇä$(*[Éç^Ññ,?DR¯É"#$1NV1MTD
>u;kxG R¯È/HN&-/@y>s>t@z]ÍìP¨À$'(D]Ïî<n|0JRU·Ò×\¼ tRNS %ñ'ïó(ò~ÑÝè IDATxC1F_Ý¿MmÛ¶4¶m{ÿ ¤nçáÓ ®A$à$b HeøTãWÄÂhh´:PtZ
Q«0@.`Þ`4-V`³Zì&A#ÁébkÝÄãñúØ>.''ø`C$FØÏ (±x"6XácTÚéL§@Iù;d d-¹|¾P,È9¡RÕÍf3¢¿F½VmM íX§ÚíÍç@Y7ÎõºÕN¬= åÈʪu
}Ö¬«+eaiq ¤Ö76íÝ=h
ûZìíî lë}á¨Ê±¥[F«I9A¹k9¥Öë ä3¢Ã9ΡóqB~Øáb¸ÃåU_¸^Ü [·ôwý{zvzÙ(£(£(þfòqÉGÉïkñÏçY¾ÿªfäòÇ~à:*4ÓQ\O> ¼<×úW£éÍZ|ÞÅ7ñïjTÔäãn½»¢®`$Hð+ò¿GOñûð*èxø¥X*|^ÿd IEND®B`PNG
IHDR @ @ ·ì :PLTE """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""%+-@yW¼×`×ø^ÑñS²ÌC,>C*8<XÁÝaÚûaÙùMµ+<AaÙúXÀÜ#%%TµÐL´=q>uK°`ÖöA}L±8do=r%+,@y^ÐïS²Ë)59=qP©ÁU·Ò"#$PªÂ\Êè0JQQªÃ"##U·Ñ#&&_Ôô>t>s`Øø_Ôó5Yc1OW5Zd1NV+=B1MU+;@/GM\Ìê*;?3S\)8<2RZ_Õõ+=A]Ïî,@F,@E&-/0KS7alO¦¾9dp8amB~EP¨ÀN¢¹'023T]]Îí?x3U^C6\gU¸Ó&.0D7_iRÆH¥I¨M ¶$(*?v
ZÆãX¿Ü-AG#$%[Éç8co[ÈæW½ÙC'25?v8bn%*+L²N£º2PX)7;=p~(58^ÒòP§¿4WaQ«ÄT´Ï0JRQ¬ÅT´ÎI¨6]hR¯ÉT³Í0LSF9eqEE9gsFC#&'\Ëé`Ö÷&/16\fBA{R®Ç]Íì(47%,.*:>*9=9fr:gt7^iU¶Ð?wZÇäX¿Û^ÑðQÅH£)6:V¹Ô'034U^E.EL.FMK®@zS× tRNS *×øÖý »½+üùóÔ,ØúôÀ=V IDATx¤ËµC! ÐïC|ÿãÚ^yR]ÕMÛáO]ßÔÕÝ0NÈ2ÍËí¿"ªª¢(0Vã(ÀY%PDT-~(m¬ó!âK ÞY£~´üIÒf{³Û Þáa¼§§ô3ÕOp& Ф¡ x÷#jôÚ¶mméòc)]m¤É)Ƨgfçhk²ñÎÒ ægg¦ÇìÐ+XÅêuiyVת·k«°²\[ü:,Ø6ØÜjIJ
;»"»;°×XùþÛfÁáÇÍûýSÎÎÏÏ8=Þo¾;æèÐ(öÓ¥BkÔeûÍ\7p+mîîáþNÚ<ÀQðOÒæùô³´yg»ttÐëoý½£ìVð»Òäsýü¬ø&_ðaüïV~à·Ö?*8àQ ;8¥Á,¸¤ f¥ 1 Üx¤×§ñ*øÑA¯Ôð°a#±³¶¦#nPi+¼¶CÈ,ÆèäÍ_áNbÑáøç HB*ÚÒ¦ L( ^<ñÃL6pJ¾PÉ¥©¢%"R,ä9Èe3eRËa1(
¢ßqÇ8Ù´mK˱mƶmÛü·yi!èΪYÏuë ÀÏ_Àï?i÷ý+òÄA|ù{´?¿_En).JËD¤<
©¬¢Z\Ts©R*( ¯©J
uX/
4J9¡5·DEµ4kÇ4&i¥V4Ú¡®Ð¯vsf:àg,¢èBC»î$¶ºÍùîá@ôI_?< ! ^ÈàÓ½ÇöäõB%Làw±FD1Á¨(F ±ø H%0Ʊ¿ÅØ(¢0ÅÄ'ÅæN.0u@íYPWìIüaNâK Ä?ðÓµ=ev/c±Ó0c0÷2Êë:06R-uÒÄ\Q̶ää´¼µ6R#
ÆF³6Òñ·rÕìuæmâðÂIñi~ Åü ÃsPþ"± ó¼
eiyå£ËPàãÊò§¡ÝÒ,S]U¦ºV
ªÖ©®Z¦êoëë·xzãâÆSnm¬{ÚºwaÙÏ
Å»´Ýõ(mg/®þå½À¿¼û[§b³µq¶Å&Õ¯¹$ñzÈH>aÌKT1/æø1O0¾.hÍYþAÓö£
-ê>Ûº«¢XÕ¢î}ߨë ÛÑ;ÃöN´ØvÅýθÿ1 ë×ÄO@&v/Äþ_ö\ôÇ\í.½+0; !fÊ ¦´Ó% JY·OÂ'/Å]_;ßÀ'"&Nªn aQ^ cx¦AáÒ IEND®B`Code explanation
In the
mindar-image-viewer.jsfile:Lines 1–6: We import essential modules
THREEfrom “three” for 3D graphics,React,useEffect, anduseReffrom “react” for component handling. TheMindARThreefrom “mind-ar/dist/mindar-image-three.prod.js” aids in AR functionality. TheGLTFLoaderfrom thethree/examples/jsm/loaders/GLTFLoader.jsloads 3D models. TheEdumarkerandEdulogopaths are imported for AR assets.Line 9: The useRef hook creates
containerRefto bind AR content to a DOM element.Lines 11–14: The
useEffecthook manages component mount/unmount tasks. It initializes aMindARThreeinstance within it, connecting content to a DOM element and defining the image target source (Edumarker).Lines 20–37: The
GLTFLoaderloads a 3D model, scales and rotates it, then adds lighting for visualization.Lines 44–47: The AR experience begins with the
mindarThree.start()and a rendering loop via therenderer.setAnimationLoop().Line 48: The
useEffecthook also provides a cleanup function for stopping AR and the render loop during unmount.
The
StartAR.jsfile creates a React component offering a UI for marker-based AR interaction. It usesuseStateto managestartedstate, displays buttons for AR control, and conditionally renders theMindARThreeViewerbased onstarted. The styling is from thestartAR.css.In
App.jsfile, a basic React app is set up with thereact-router-domfor routing. Modules are imported, App component defined, and a single route rendersStartARfor path “/.” The, theAppis exported for use.
Output
The output of the above application looks something like this:
There are plenty of use cases for MindAR image tracking that could add gamification to otherwise monotonous apps or content. For example, we could use the application to display food over a restaurant menu.
Free Resources