1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
|
<!DOCTYPE html>
<html lang="en">
<head>
<link rel="stylesheet" href="https://unpkg.com/latex.css/style.min.css" />
<link rel="stylesheet" href="/assets/main.css" />
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Introduction to AR.js and Natural Feature Tracking</title>
<meta name="og:site_name" content="Navan Chauhan" />
<link rel="canonical" href="https://web.navan.dev/posts/2020-08-01-Natural-Feature-Tracking-ARJS.html" />
<meta name="twitter:url" content="https://web.navan.dev/posts/2020-08-01-Natural-Feature-Tracking-ARJS.html />
<meta name="og:url" content="https://web.navan.dev/posts/2020-08-01-Natural-Feature-Tracking-ARJS.html" />
<meta name="twitter:title" content="Introduction to AR.js and Natural Feature Tracking" />
<meta name="og:title" content="Introduction to AR.js and Natural Feature Tracking" />
<meta name="description" content="An introduction to AR.js and NFT" />
<meta name="twitter:description" content="An introduction to AR.js and NFT" />
<meta name="og:description" content="An introduction to AR.js and NFT" />
<meta name="twitter:card" content="summary_large_image" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<link rel="shortcut icon" href="/images/favicon.png" type="image/png" />
<link rel="alternate" href="/feed.rss" type="application/rss+xml" title="Subscribe to Navan Chauhan" />
<meta name="twitter:image" content="https://web.navan.dev/images/opengraph/posts/2020-08-01-Natural-Feature-Tracking-ARJS.png" />
<meta name="og:image" content="https://web.navan.dev/images/opengraph/posts/2020-08-01-Natural-Feature-Tracking-ARJS.png" />
<meta name="google-site-verification" content="LVeSZxz-QskhbEjHxOi7-BM5dDxTg53x2TwrjFxfL0k" />
<script data-goatcounter="https://navanchauhan.goatcounter.com/count"
async src="//gc.zgo.at/count.js"></script>
<script defer data-domain="web.navan.dev" src="https://plausible.io/js/plausible.js"></script>
<link rel="manifest" href="/manifest.json" />
</head>
<body>
<center><nav style="display: block;">
|
<a href="/">home</a> |
<a href="/about/">about/links</a> |
<a href="/posts/">posts</a> |
<a href="/3D-Designs/">3D designs</a> |
<!--<a href="/publications/">publications</a> |-->
<!--<a href="/repo/">iOS repo</a> |-->
<a href="/feed.rss">RSS Feed</a> |
</nav>
</center>
<main>
<h1 id="introduction-to-arjs-and-natural-feature-tracking">Introduction to AR.js and Natural Feature Tracking</h1>
<h2 id="arjs">AR.js</h2>
<p>AR.js is a lightweight library for Augmented Reality on the Web, coming with features like Image Tracking, Location based AR and Marker tracking. It is the easiest option for cross-browser augmented reality. </p>
<p>The same code works for iOS, Android, Desktops and even VR Browsers!</p>
<p>It was initially created by Jerome Etienne and is now maintained by Nicolo Carpignoli and the AR-js Organisation</p>
<h2 id="nft">NFT</h2>
<p>Usually for augmented reality you need specialised markers, like this Hiro marker (notice the thick non-aesthetic borders 🤢)</p>
<p><img src="https://upload.wikimedia.org/wikipedia/commons/4/48/Hiro_marker_ARjs.png" alt="" /></p>
<p>This is called marker based tracking where the code knows what to look for. NFT or Natural Feature Tracing converts normal images into markers by extracting 'features' from it, this way you can use any image of your liking!</p>
<p>I'll be using my GitHub profile picture </p>
<p><img src="/images/me.jpeg" alt="" /></p>
<h2 id="creating-the-marker">Creating the Marker!</h2>
<p>First we need to create the marker files required by AR.js for NFT. For this we use Carnaux's repository 'NFT-Marker-Creator'.</p>
<pre><code>$ git clone https://github.com/Carnaux/NFT-Marker-Creator
Cloning into 'NFT-Marker-Creator'...
remote: Enumerating objects: 79, done.
remote: Counting objects: 100% (79/79), done.
remote: Compressing objects: 100% (72/72), done.
remote: Total 580 (delta 10), reused 59 (delta 7), pack-reused 501
Receiving objects: 100% (580/580), 9.88 MiB | 282.00 KiB/s, done.
Resolving deltas: 100% (262/262), done.
$ cd NFT-Makrer-Creator
</code></pre>
<h3 id="install-the-dependencies">Install the dependencies</h3>
<pre><code>$ npm install
npm WARN nodegenerator@1.0.0 No repository field.
added 67 packages from 56 contributors and audited 67 packages in 2.96s
1 package is looking for funding
run `npm fund` for details
found 0 vulnerabilities
╭────────────────────────────────────────────────────────────────╮
│ │
│ New patch version of npm available! 6.14.5 → 6.14.7 │
│ Changelog: https://github.com/npm/cli/releases/tag/v6.14.7 │
│ Run npm install -g npm to update! │
│ │
╰────────────────────────────────────────────────────────────────╯
</code></pre>
<h3 id="copy-the-target-marker-to-the-folder">Copy the target marker to the folder</h3>
<pre><code>$ cp ~/CodingAndStuff/ARjs/me.png .
</code></pre>
<h3 id="generate-marker">Generate Marker</h3>
<pre><code>$ node app.js -i me.png
Confidence level: [ * * * * * ] 5/5 || Entropy: 5.24 || Current max: 5.17 min: 4.6
Do you want to continue? (Y/N)
y
writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!
[info]
Commands:
[info] --
Generator started at 2020-08-01 16:01:41 +0580
[info] Tracking Extraction Level = 2
[info] MAX_THRESH = 0.900000
[info] MIN_THRESH = 0.550000
[info] SD_THRESH = 8.000000
[info] Initialization Extraction Level = 1
[info] SURF_FEATURE = 100
[info] min allow 3.699000.
[info] Image DPI (1): 3.699000
[info] Image DPI (2): 4.660448
[info] Image DPI (3): 5.871797
[info] Image DPI (4): 7.398000
[info] Image DPI (5): 9.320896
[info] Image DPI (6): 11.743593
[info] Image DPI (7): 14.796000
[info] Image DPI (8): 18.641792
[info] Image DPI (9): 23.487186
[info] Image DPI (10): 29.592001
[info] Image DPI (11): 37.283585
[info] Image DPI (12): 46.974373
[info] Image DPI (13): 59.184002
[info] Image DPI (14): 72.000000
[info] Generating ImageSet...
[info] (Source image xsize=568, ysize=545, channels=3, dpi=72.0).
[info] Done.
[info] Saving to asa.iset...
[info] Done.
[info] Generating FeatureList...
...
[info] (46, 44) 5.871797[dpi]
[info] Freak features - 23[info] ========= 23 ===========
[info] (37, 35) 4.660448[dpi]
[info] Freak features - 19[info] ========= 19 ===========
[info] (29, 28) 3.699000[dpi]
[info] Freak features - 9[info] ========= 9 ===========
[info] Done.
[info] Saving FeatureSet3...
[info] Done.
[info] Generator finished at 2020-08-01 16:02:02 +0580
--
Finished marker creation!
Now configuring demo!
Finished!
To run demo use: 'npm run demo'
</code></pre>
<p>Now we have the required files in the output folder</p>
<pre><code>$ ls output
me.fset me.fset3 me.iset
</code></pre>
<h2 id="creating-the-html-page">Creating the HTML Page</h2>
<p>Create a new file called <code>index.html</code> in your project folder. This is the basic template we are going to use. Replace <code>me</code> with the root filename of your image, for example <code>NeverGonnaGiveYouUp.png</code> will become <code>NeverGonnaGiveYouUp</code>. Make sure you have copied all three files from the output folder in the previous step to the root of your project folder.</p>
<div class="codehilite">
<pre><span></span><code><span class="p"><</span><span class="nt">script</span> <span class="na">src</span><span class="o">=</span><span class="s">"https://cdn.jsdelivr.net/gh/aframevr/aframe@1c2407b26c61958baa93967b5412487cd94b290b/dist/aframe-master.min.js"</span><span class="p">></</span><span class="nt">script</span><span class="p">></span>
<span class="p"><</span><span class="nt">script</span> <span class="na">src</span><span class="o">=</span><span class="s">"https://raw.githack.com/AR-js-org/AR.js/master/aframe/build/aframe-ar-nft.js"</span><span class="p">></</span><span class="nt">script</span><span class="p">></span>
<span class="p"><</span><span class="nt">style</span><span class="p">></span>
<span class="w"> </span><span class="p">.</span><span class="nc">arjs-loader</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">height</span><span class="p">:</span><span class="w"> </span><span class="mi">100</span><span class="kt">%</span><span class="p">;</span>
<span class="w"> </span><span class="k">width</span><span class="p">:</span><span class="w"> </span><span class="mi">100</span><span class="kt">%</span><span class="p">;</span>
<span class="w"> </span><span class="k">position</span><span class="p">:</span><span class="w"> </span><span class="kc">absolute</span><span class="p">;</span>
<span class="w"> </span><span class="k">top</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span>
<span class="w"> </span><span class="k">left</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span>
<span class="w"> </span><span class="k">background-color</span><span class="p">:</span><span class="w"> </span><span class="nb">rgba</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="mf">0.8</span><span class="p">);</span>
<span class="w"> </span><span class="k">z-index</span><span class="p">:</span><span class="w"> </span><span class="mi">9999</span><span class="p">;</span>
<span class="w"> </span><span class="k">display</span><span class="p">:</span><span class="w"> </span><span class="kc">flex</span><span class="p">;</span>
<span class="w"> </span><span class="k">justify-content</span><span class="p">:</span><span class="w"> </span><span class="kc">center</span><span class="p">;</span>
<span class="w"> </span><span class="k">align-items</span><span class="p">:</span><span class="w"> </span><span class="kc">center</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">.</span><span class="nc">arjs-loader</span><span class="w"> </span><span class="nt">div</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">text-align</span><span class="p">:</span><span class="w"> </span><span class="kc">center</span><span class="p">;</span>
<span class="w"> </span><span class="k">font-size</span><span class="p">:</span><span class="w"> </span><span class="mf">1.25</span><span class="kt">em</span><span class="p">;</span>
<span class="w"> </span><span class="k">color</span><span class="p">:</span><span class="w"> </span><span class="kc">white</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="p"></</span><span class="nt">style</span><span class="p">></span>
<span class="p"><</span><span class="nt">body</span> <span class="na">style</span><span class="o">=</span><span class="s">"margin : 0px; overflow: hidden;"</span><span class="p">></span>
<span class="p"><</span><span class="nt">div</span> <span class="na">class</span><span class="o">=</span><span class="s">"arjs-loader"</span><span class="p">></span>
<span class="p"><</span><span class="nt">div</span><span class="p">></span>Calculating Image Descriptors....<span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nt">a-scene</span>
<span class="na">vr-mode-ui</span><span class="o">=</span><span class="s">"enabled: false;"</span>
<span class="na">renderer</span><span class="o">=</span><span class="s">"logarithmicDepthBuffer: true;"</span>
<span class="na">embedded</span>
<span class="na">arjs</span><span class="o">=</span><span class="s">"trackingMethod: best; sourceType: webcam;debugUIEnabled: false;"</span>
<span class="p">></span>
<span class="p"><</span><span class="nt">a-nft</span>
<span class="na">type</span><span class="o">=</span><span class="s">"nft"</span>
<span class="na">url</span><span class="o">=</span><span class="s">"./me"</span>
<span class="na">smooth</span><span class="o">=</span><span class="s">"true"</span>
<span class="na">smoothCount</span><span class="o">=</span><span class="s">"10"</span>
<span class="na">smoothTolerance</span><span class="o">=</span><span class="s">".01"</span>
<span class="na">smoothThreshold</span><span class="o">=</span><span class="s">"5"</span>
<span class="p">></span>
<span class="p"></</span><span class="nt">a-nft</span><span class="p">></span>
<span class="p"><</span><span class="nt">a-entity</span> <span class="na">camera</span><span class="p">></</span><span class="nt">a-entity</span><span class="p">></span>
<span class="p"></</span><span class="nt">a-scene</span><span class="p">></span>
<span class="p"></</span><span class="nt">body</span><span class="p">></span>
</code></pre>
</div>
<p>In this we are creating a AFrame scene and we are telling it that we want to use NFT Tracking. The amazing part about using AFrame is that we are able to use all AFrame objects!</p>
<h2 id="adding-a-simple-box">Adding a simple box</h2>
<p>Let us add a simple box!</p>
<div class="codehilite">
<pre><span></span><code><span class="p"><</span><span class="nt">a-nft</span> <span class="err">.....</span><span class="p">></span>
<span class="p"><</span><span class="nt">a-box</span> <span class="na">position</span><span class="o">=</span><span class="s">'100 0.5 -180'</span> <span class="na">material</span><span class="o">=</span><span class="s">'opacity: 0.5; side: double'</span> <span class="na">scale</span><span class="o">=</span><span class="s">"100 100 100"</span><span class="p">></</span><span class="nt">a-box</span><span class="p">></span>
<span class="p"></</span><span class="nt">a-nft</span><span class="p">></span>
</code></pre>
</div>
<p>Now to test it out we will need to create a simple server, I use Python's inbuilt <code>SimpleHTTPServer</code> alongside <code>ngrok</code> </p>
<p>In one terminal window, <code>cd</code> to the project directory. Currently your project folder should have 4 files, <code>index.html</code>, <code>me.fset3</code>, <code>me.fset</code> and <code>me.iset</code></p>
<p>Open up two terminal windows and <code>cd</code> into your project folder then run the following commands to start up your server.</p>
<p>In the first terminal window start the Python Server</p>
<pre><code>$ cd ~/CodingAndStuff/ARjs
$ python2 -m SimpleHTTPServer
Serving HTTP on 0.0.0.0 port 8000 ...
</code></pre>
<p>In the other window run <code>ngrok</code> ( Make sure you have installed it prior to running this step )</p>
<pre><code>$ ngrok http 8000
</code></pre>
<p><img src="/assets/posts/arjs/01-ngrok.png" alt="" /></p>
<p>Now copy the url to your phone and try running the example </p>
<p><img src="/assets/posts/arjs/02-box-demo.gif" alt="" /></p>
<p>👏 Congratulations! You just built an Augmented Reality experience using AR.js and AFrame</p>
<h2 id="adding-a-torus-knot-in-the-box">Adding a Torus-Knot in the box</h2>
<p>Edit your <code>index.html</code> </p>
<div class="codehilite">
<pre><span></span><code><span class="p"><</span><span class="nt">a-nft</span> <span class="err">..</span><span class="p">></span>
<span class="p"><</span><span class="nt">a-box</span> <span class="err">..</span><span class="p">></span>
<span class="p"><</span><span class="nt">a-torus-knot</span> <span class="na">radius</span><span class="o">=</span><span class="s">'0.26'</span> <span class="na">radius-tubular</span><span class="o">=</span><span class="s">'0.05'</span> <span class="p">></</span><span class="nt">a-torus-knot</span><span class="p">></span>
<span class="p"></</span> <span class="nt">a-box</span><span class="p">></span>
<span class="p"></</span> <span class="nt">a-nft</span><span class="p">></span>
</code></pre>
</div>
<p><img src="/assets/posts/arjs/03-knot.png" alt="" /></p>
<h2 id="where-are-the-gifs">Where are the GIFs?</h2>
<p>Now that we know how to place a box in the scene and add a torus knot in it, what do we do next? We bring the classic internet back!</p>
<p><code>AFrame GIF Shader</code> is a gif shader for A-Frame created by mayognaise.</p>
<h3 id="first-things-first">First things first</h3>
<p>Add <code><script src="https://rawgit.com/mayognaise/aframe-gif-shader/master/dist/aframe-gif-shader.min.js"></script></code> to <code><head></code></p>
<p>Change the box's material to add the GIF shader</p>
<div class="codehilite">
<pre><span></span><code>...
<span class="p"><</span><span class="nt">a-box</span> <span class="na">position</span><span class="o">=</span><span class="s">'100 0.5 -180'</span> <span class="na">material</span><span class="o">=</span><span class="s">"shader:gif;src:url(https://media.tenor.com/images/412b1aa9149d98d561df62db221e0789/tenor.gif);opacity:.5"</span> <span class="err">.....</span><span class="p">></span>
</code></pre>
</div>
<p><img src="/assets/posts/arjs/04-nyan.gif" alt="" /></p>
<h2 id="bonus-idea-integrate-it-with-githubs-new-profile-readme-feature">Bonus Idea: Integrate it with GitHub's new profile Readme Feature!</h2>
<h3 id="1-host-the-code-using-github-pages">1) Host the code using GitHub Pages</h3>
<h3 id="2-create-a-new-repository-the-name-should-be-your-github-username">2) Create a new repository ( the name should be your GitHub username )</h3>
<h3 id="3-add-qr-code-to-the-page-and-tell-the-users-to-scan-your-profile-picture">3) Add QR Code to the page and tell the users to scan your profile picture</h3>
<h3 id="profit">??) Profit 💸</h3>
<p>Here is a screenshot of me scanning a rounded version of my profile picture ( It still works! Even though the image is cropped and I haven't changed any line of code )</p>
<p><img src="/assets/posts/arjs/05-GitHub.jpg" alt="" /></p>
<blockquote>If you have scrolled this far, consider subscribing to my mailing list <a href="https://listmonk.navan.dev/subscription/form">here.</a> You can subscribe to either a specific type of post you are interested in, or subscribe to everything with the "Everything" list.</blockquote>
<script data-isso="https://comments.navan.dev/"
src="https://comments.navan.dev/js/embed.min.js"></script>
<section id="isso-thread">
<noscript>Javascript needs to be activated to view comments.</noscript>
</section>
</main>
<script src="assets/manup.min.js"></script>
<script src="/pwabuilder-sw-register.js"></script>
</body>
</html>
|
