diff options
Diffstat (limited to 'docs/posts/2020-01-16-Image-Classifier-Using-Turicreate.html')
| -rw-r--r-- | docs/posts/2020-01-16-Image-Classifier-Using-Turicreate.html | 276 |
1 files changed, 276 insertions, 0 deletions
diff --git a/docs/posts/2020-01-16-Image-Classifier-Using-Turicreate.html b/docs/posts/2020-01-16-Image-Classifier-Using-Turicreate.html new file mode 100644 index 0000000..f149a5c --- /dev/null +++ b/docs/posts/2020-01-16-Image-Classifier-Using-Turicreate.html @@ -0,0 +1,276 @@ +<!DOCTYPE html> +<html lang="en"> +<head> + + <link rel="stylesheet" href="/assets/main.css" /> + <link rel="stylesheet" href="/assets/sakura.css" /> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + <title>Hey - Post</title> + +</head> +<body> + <nav style="display: block;"> +| +<a href="/">home</a> | +<a href="/about/">about/links</a> | +<a href="/posts/">posts</a> | +<a href="/publications/">publications</a> | +<a href="/repo/">iOS repo</a> | +<a href="/feed.rss">RSS Feed</a> | +</nav> + +<main> + <h1>Creating a Custom Image Classifier using Turicreate to detect Smoke and Fire</h1> + +<p><em>For setting up Kaggle with Google Colab, please refer to <a rel="noopener" target="_blank" href="/posts/2020-01-15-Setting-up-Kaggle-to-use-with-Colab/"> my previous post</a></em></p> + +<h2>Dataset</h2> + +<h3>Mounting Google Drive</h3> + +<div class="codehilite"><pre><span></span><code><span class="kn">import</span> <span class="nn">os</span> +<span class="kn">from</span> <span class="nn">google.colab</span> <span class="kn">import</span> <span class="n">drive</span> +<span class="n">drive</span><span class="o">.</span><span class="n">mount</span><span class="p">(</span><span class="s1">'/content/drive'</span><span class="p">)</span> +</code></pre></div> + +<h3>Downloading Dataset from Kaggle</h3> + +<div class="codehilite"><pre><span></span><code><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s1">'KAGGLE_CONFIG_DIR'</span><span class="p">]</span> <span class="o">=</span> <span class="s2">"/content/drive/My Drive/"</span> +<span class="err">!</span><span class="n">kaggle</span> <span class="n">datasets</span> <span class="n">download</span> <span class="n">ashutosh69</span><span class="o">/</span><span class="n">fire</span><span class="o">-</span><span class="ow">and</span><span class="o">-</span><span class="n">smoke</span><span class="o">-</span><span class="n">dataset</span> +<span class="err">!</span><span class="n">unzip</span> <span class="s2">"fire-and-smoke-dataset.zip"</span> +</code></pre></div> + +<h2>Pre-Processing</h2> + +<div class="codehilite"><pre><span></span><code><span class="nt">!mkdir</span><span class="na"> default smoke fire</span> +</code></pre></div> + +<p>\</p> + +<div class="codehilite"><pre><span></span><code><span class="nt">!ls</span><span class="na"> data/data/img_data/train/default/*.jpg</span> +</code></pre></div> + +<p>\</p> + +<div class="codehilite"><pre><span></span><code><span class="nt">img_1002.jpg</span><span class="na"> img_20.jpg img_519.jpg img_604.jpg img_80.jpg</span> +<span class="na">img_1003.jpg img_21.jpg img_51.jpg img_60.jpg img_8.jpg</span> +<span class="na">img_1007.jpg img_22.jpg img_520.jpg img_61.jpg img_900.jpg</span> +<span class="na">img_100.jpg img_23.jpg img_521.jpg 'img_62 (2).jpg' img_920.jpg</span> +<span class="na">img_1014.jpg img_24.jpg 'img_52 (2).jpg' img_62.jpg img_921.jpg</span> +<span class="na">img_1018.jpg img_29.jpg img_522.jpg 'img_63 (2).jpg' img_922.jpg</span> +<span class="na">img_101.jpg img_3000.jpg img_523.jpg img_63.jpg img_923.jpg</span> +<span class="na">img_1027.jpg img_335.jpg img_524.jpg img_66.jpg img_924.jpg</span> +<span class="na">img_102.jpg img_336.jpg img_52.jpg img_67.jpg img_925.jpg</span> +<span class="na">img_1042.jpg img_337.jpg img_530.jpg img_68.jpg img_926.jpg</span> +<span class="na">img_1043.jpg img_338.jpg img_531.jpg img_700.jpg img_927.jpg</span> +<span class="na">img_1046.jpg img_339.jpg 'img_53 (2).jpg' img_701.jpg img_928.jpg</span> +<span class="na">img_1052.jpg img_340.jpg img_532.jpg img_702.jpg img_929.jpg</span> +<span class="na">img_107.jpg img_341.jpg img_533.jpg img_703.jpg img_930.jpg</span> +<span class="na">img_108.jpg img_3.jpg img_537.jpg img_704.jpg img_931.jpg</span> +<span class="na">img_109.jpg img_400.jpg img_538.jpg img_705.jpg img_932.jpg</span> +<span class="na">img_10.jpg img_471.jpg img_539.jpg img_706.jpg img_933.jpg</span> +<span class="na">img_118.jpg img_472.jpg img_53.jpg img_707.jpg img_934.jpg</span> +<span class="na">img_12.jpg img_473.jpg img_540.jpg img_708.jpg img_935.jpg</span> +<span class="na">img_14.jpg img_488.jpg img_541.jpg img_709.jpg img_938.jpg</span> +<span class="na">img_15.jpg img_489.jpg 'img_54 (2).jpg' img_70.jpg img_958.jpg</span> +<span class="na">img_16.jpg img_490.jpg img_542.jpg img_710.jpg img_971.jpg</span> +<span class="na">img_17.jpg img_491.jpg img_543.jpg 'img_71 (2).jpg' img_972.jpg</span> +<span class="na">img_18.jpg img_492.jpg img_54.jpg img_71.jpg img_973.jpg</span> +<span class="na">img_19.jpg img_493.jpg 'img_55 (2).jpg' img_72.jpg img_974.jpg</span> +<span class="na">img_1.jpg img_494.jpg img_55.jpg img_73.jpg img_975.jpg</span> +<span class="na">img_200.jpg img_495.jpg img_56.jpg img_74.jpg img_980.jpg</span> +<span class="na">img_201.jpg img_496.jpg img_57.jpg img_75.jpg img_988.jpg</span> +<span class="na">img_202.jpg img_497.jpg img_58.jpg img_76.jpg img_9.jpg</span> +<span class="na">img_203.jpg img_4.jpg img_59.jpg img_77.jpg</span> +<span class="na">img_204.jpg img_501.jpg img_601.jpg img_78.jpg</span> +<span class="na">img_205.jpg img_502.jpg img_602.jpg img_79.jpg</span> +<span class="na">img_206.jpg img_50.jpg img_603.jpg img_7.jpg</span> +</code></pre></div> + +<p>The image files are not actually JPEG, thus we first need to save them in the correct format for Turicreate</p> + +<div class="codehilite"><pre><span></span><code><span class="kn">from</span> <span class="nn">PIL</span> <span class="kn">import</span> <span class="n">Image</span> +<span class="kn">import</span> <span class="nn">glob</span> + + +<span class="n">folders</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"default"</span><span class="p">,</span><span class="s2">"smoke"</span><span class="p">,</span><span class="s2">"fire"</span><span class="p">]</span> +<span class="k">for</span> <span class="n">folder</span> <span class="ow">in</span> <span class="n">folders</span><span class="p">:</span> + <span class="n">n</span> <span class="o">=</span> <span class="mi">1</span> + <span class="k">for</span> <span class="n">file</span> <span class="ow">in</span> <span class="n">glob</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s2">"./data/data/img_data/train/"</span> <span class="o">+</span> <span class="n">folder</span> <span class="o">+</span> <span class="s2">"/*.jpg"</span><span class="p">):</span> + <span class="n">im</span> <span class="o">=</span> <span class="n">Image</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="n">file</span><span class="p">)</span> + <span class="n">rgb_im</span> <span class="o">=</span> <span class="n">im</span><span class="o">.</span><span class="n">convert</span><span class="p">(</span><span class="s1">'RGB'</span><span class="p">)</span> + <span class="n">rgb_im</span><span class="o">.</span><span class="n">save</span><span class="p">((</span><span class="n">folder</span> <span class="o">+</span> <span class="s2">"/"</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">n</span><span class="p">)</span> <span class="o">+</span> <span class="s2">".jpg"</span><span class="p">),</span> <span class="n">quality</span><span class="o">=</span><span class="mi">100</span><span class="p">)</span> + <span class="n">n</span> <span class="o">+=</span><span class="mi">1</span> + <span class="k">for</span> <span class="n">file</span> <span class="ow">in</span> <span class="n">glob</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s2">"./data/data/img_data/train/"</span> <span class="o">+</span> <span class="n">folder</span> <span class="o">+</span> <span class="s2">"/*.jpg"</span><span class="p">):</span> + <span class="n">im</span> <span class="o">=</span> <span class="n">Image</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="n">file</span><span class="p">)</span> + <span class="n">rgb_im</span> <span class="o">=</span> <span class="n">im</span><span class="o">.</span><span class="n">convert</span><span class="p">(</span><span class="s1">'RGB'</span><span class="p">)</span> + <span class="n">rgb_im</span><span class="o">.</span><span class="n">save</span><span class="p">((</span><span class="n">folder</span> <span class="o">+</span> <span class="s2">"/"</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">n</span><span class="p">)</span> <span class="o">+</span> <span class="s2">".jpg"</span><span class="p">),</span> <span class="n">quality</span><span class="o">=</span><span class="mi">100</span><span class="p">)</span> + <span class="n">n</span> <span class="o">+=</span><span class="mi">1</span> +</code></pre></div> + +<p>\</p> + +<div class="codehilite"><pre><span></span><code><span class="nt">!mkdir</span><span class="na"> train</span> +<span class="na">!mv default ./train</span> +<span class="na">!mv smoke ./train</span> +<span class="na">!mv fire ./train</span> +</code></pre></div> + +<h2>Making the Image Classifier</h2> + +<h3>Making an SFrame</h3> + +<div class="codehilite"><pre><span></span><code><span class="nt">!pip</span><span class="na"> install turicreate</span> +</code></pre></div> + +<p>\</p> + +<div class="codehilite"><pre><span></span><code><span class="kn">import</span> <span class="nn">turicreate</span> <span class="k">as</span> <span class="nn">tc</span> +<span class="kn">import</span> <span class="nn">os</span> + +<span class="n">data</span> <span class="o">=</span> <span class="n">tc</span><span class="o">.</span><span class="n">image_analysis</span><span class="o">.</span><span class="n">load_images</span><span class="p">(</span><span class="s2">"./train"</span><span class="p">,</span> <span class="n">with_path</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> + +<span class="n">data</span><span class="p">[</span><span class="s2">"label"</span><span class="p">]</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="s2">"path"</span><span class="p">]</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">path</span><span class="p">:</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">basename</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">dirname</span><span class="p">(</span><span class="n">path</span><span class="p">)))</span> + +<span class="nb">print</span><span class="p">(</span><span class="n">data</span><span class="p">)</span> + +<span class="n">data</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="s1">'fire-smoke.sframe'</span><span class="p">)</span> +</code></pre></div> + +<p>\</p> + +<div class="codehilite"><pre><span></span><code><span class="nt">+-------------------------+------------------------+</span> +<span class="err">| path | image |</span> +<span class="nt">+-------------------------+------------------------+</span> +<span class="err">| ./train/default/1.jpg | Height: 224 Width: 224 |</span> +<span class="err">| ./train/default/10.jpg | Height: 224 Width: 224 |</span> +<span class="err">| ./train/default/100.jpg | Height: 224 Width: 224 |</span> +<span class="err">| ./train/default/101.jpg | Height: 224 Width: 224 |</span> +<span class="err">| ./train/default/102.jpg | Height: 224 Width: 224 |</span> +<span class="err">| ./train/default/103.jpg | Height: 224 Width: 224 |</span> +<span class="err">| ./train/default/104.jpg | Height: 224 Width: 224 |</span> +<span class="err">| ./train/default/105.jpg | Height: 224 Width: 224 |</span> +<span class="err">| ./train/default/106.jpg | Height: 224 Width: 224 |</span> +<span class="err">| ./train/default/107.jpg | Height: 224 Width: 224 |</span> +<span class="nt">+-------------------------+------------------------+</span> +<span class="nt">[2028</span><span class="na"> rows x 2 columns]</span> +<span class="na">Note</span><span class="p">:</span><span class="err"> </span><span class="nc">Only</span><span class="err"> </span><span class="nc">the</span><span class="err"> </span><span class="nc">head</span><span class="err"> </span><span class="nc">of</span><span class="err"> </span><span class="nc">the</span><span class="err"> </span><span class="nc">SFrame</span><span class="err"> </span><span class="nc">is</span><span class="err"> </span><span class="nc">printed.</span> +<span class="nt">You</span><span class="na"> can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.</span> +<span class="na">+-------------------------+------------------------+---------+</span> +<span class="p">|</span><span class="na"> path </span><span class="p">|</span><span class="na"> image </span><span class="p">|</span><span class="na"> label </span><span class="p">|</span> +<span class="nt">+-------------------------+------------------------+---------+</span> +<span class="err">| ./train/default/1.jpg | Height: 224 Width: 224 | default |</span> +<span class="err">| ./train/default/10.jpg | Height: 224 Width: 224 | default |</span> +<span class="err">| ./train/default/100.jpg | Height: 224 Width: 224 | default |</span> +<span class="err">| ./train/default/101.jpg | Height: 224 Width: 224 | default |</span> +<span class="err">| ./train/default/102.jpg | Height: 224 Width: 224 | default |</span> +<span class="err">| ./train/default/103.jpg | Height: 224 Width: 224 | default |</span> +<span class="err">| ./train/default/104.jpg | Height: 224 Width: 224 | default |</span> +<span class="err">| ./train/default/105.jpg | Height: 224 Width: 224 | default |</span> +<span class="err">| ./train/default/106.jpg | Height: 224 Width: 224 | default |</span> +<span class="err">| ./train/default/107.jpg | Height: 224 Width: 224 | default |</span> +<span class="nt">+-------------------------+------------------------+---------+</span> +<span class="nt">[2028</span><span class="na"> rows x 3 columns]</span> +<span class="na">Note</span><span class="p">:</span><span class="err"> </span><span class="nc">Only</span><span class="err"> </span><span class="nc">the</span><span class="err"> </span><span class="nc">head</span><span class="err"> </span><span class="nc">of</span><span class="err"> </span><span class="nc">the</span><span class="err"> </span><span class="nc">SFrame</span><span class="err"> </span><span class="nc">is</span><span class="err"> </span><span class="nc">printed.</span> +<span class="nt">You</span><span class="na"> can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.</span> +</code></pre></div> + +<h3>Making the Model</h3> + +<div class="codehilite"><pre><span></span><code><span class="kn">import</span> <span class="nn">turicreate</span> <span class="k">as</span> <span class="nn">tc</span> + +<span class="c1"># Load the data</span> +<span class="n">data</span> <span class="o">=</span> <span class="n">tc</span><span class="o">.</span><span class="n">SFrame</span><span class="p">(</span><span class="s1">'fire-smoke.sframe'</span><span class="p">)</span> + +<span class="c1"># Make a train-test split</span> +<span class="n">train_data</span><span class="p">,</span> <span class="n">test_data</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">random_split</span><span class="p">(</span><span class="mf">0.8</span><span class="p">)</span> + +<span class="c1"># Create the model</span> +<span class="n">model</span> <span class="o">=</span> <span class="n">tc</span><span class="o">.</span><span class="n">image_classifier</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="n">train_data</span><span class="p">,</span> <span class="n">target</span><span class="o">=</span><span class="s1">'label'</span><span class="p">)</span> + +<span class="c1"># Save predictions to an SArray</span> +<span class="n">predictions</span> <span class="o">=</span> <span class="n">model</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">test_data</span><span class="p">)</span> + +<span class="c1"># Evaluate the model and print the results</span> +<span class="n">metrics</span> <span class="o">=</span> <span class="n">model</span><span class="o">.</span><span class="n">evaluate</span><span class="p">(</span><span class="n">test_data</span><span class="p">)</span> +<span class="nb">print</span><span class="p">(</span><span class="n">metrics</span><span class="p">[</span><span class="s1">'accuracy'</span><span class="p">])</span> + +<span class="c1"># Save the model for later use in Turi Create</span> +<span class="n">model</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="s1">'fire-smoke.model'</span><span class="p">)</span> + +<span class="c1"># Export for use in Core ML</span> +<span class="n">model</span><span class="o">.</span><span class="n">export_coreml</span><span class="p">(</span><span class="s1">'fire-smoke.mlmodel'</span><span class="p">)</span> +</code></pre></div> + +<p>\</p> + +<div class="codehilite"><pre><span></span><code><span class="nt">Performing</span><span class="na"> feature extraction on resized images...</span> +<span class="na">Completed 64/1633</span> +<span class="na">Completed 128/1633</span> +<span class="na">Completed 192/1633</span> +<span class="na">Completed 256/1633</span> +<span class="na">Completed 320/1633</span> +<span class="na">Completed 384/1633</span> +<span class="na">Completed 448/1633</span> +<span class="na">Completed 512/1633</span> +<span class="na">Completed 576/1633</span> +<span class="na">Completed 640/1633</span> +<span class="na">Completed 704/1633</span> +<span class="na">Completed 768/1633</span> +<span class="na">Completed 832/1633</span> +<span class="na">Completed 896/1633</span> +<span class="na">Completed 960/1633</span> +<span class="na">Completed 1024/1633</span> +<span class="na">Completed 1088/1633</span> +<span class="na">Completed 1152/1633</span> +<span class="na">Completed 1216/1633</span> +<span class="na">Completed 1280/1633</span> +<span class="na">Completed 1344/1633</span> +<span class="na">Completed 1408/1633</span> +<span class="na">Completed 1472/1633</span> +<span class="na">Completed 1536/1633</span> +<span class="na">Completed 1600/1633</span> +<span class="na">Completed 1633/1633</span> +<span class="na">PROGRESS</span><span class="p">:</span><span class="err"> </span><span class="nc">Creating</span><span class="err"> </span><span class="nc">a</span><span class="err"> </span><span class="nc">validation</span><span class="err"> </span><span class="nc">set</span><span class="err"> </span><span class="nc">from</span><span class="err"> </span><span class="nc">5</span><span class="err"> </span><span class="nc">percent</span><span class="err"> </span><span class="nc">of</span><span class="err"> </span><span class="nc">training</span><span class="err"> </span><span class="nc">data.</span><span class="err"> </span><span class="nc">This</span><span class="err"> </span><span class="nc">may</span><span class="err"> </span><span class="nc">take</span><span class="err"> </span><span class="nc">a</span><span class="err"> </span><span class="nc">while.</span> + <span class="err">You can set ``validation_set=None`` to disable validation tracking.</span> + +<span class="nt">Logistic</span><span class="na"> regression</span><span class="p">:</span> +<span class="nt">--------------------------------------------------------</span> +<span class="nt">Number</span><span class="na"> of examples </span><span class="p">:</span><span class="err"> </span><span class="nc">1551</span> +<span class="nt">Number</span><span class="na"> of classes </span><span class="p">:</span><span class="err"> </span><span class="nc">3</span> +<span class="nt">Number</span><span class="na"> of feature columns </span><span class="p">:</span><span class="err"> </span><span class="nc">1</span> +<span class="nt">Number</span><span class="na"> of unpacked features </span><span class="p">:</span><span class="err"> </span><span class="nc">2048</span> +<span class="nt">Number</span><span class="na"> of coefficients </span><span class="p">:</span><span class="err"> </span><span class="nc">4098</span> +<span class="nt">Starting</span><span class="na"> L-BFGS</span> +<span class="na">--------------------------------------------------------</span> +<span class="na">+-----------+----------+-----------+--------------+-------------------+---------------------+</span> +<span class="p">|</span><span class="na"> Iteration </span><span class="p">|</span><span class="na"> Passes </span><span class="p">|</span><span class="na"> Step size </span><span class="p">|</span><span class="na"> Elapsed Time </span><span class="p">|</span><span class="na"> Training Accuracy </span><span class="p">|</span><span class="na"> Validation Accuracy </span><span class="p">|</span> +<span class="nt">+-----------+----------+-----------+--------------+-------------------+---------------------+</span> +<span class="err">| 0 | 6 | 0.018611 | 0.891830 | 0.553836 | 0.560976 |</span> +<span class="err">| 1 | 10 | 0.390832 | 1.622383 | 0.744681 | 0.792683 |</span> +<span class="err">| 2 | 11 | 0.488541 | 1.943987 | 0.733075 | 0.804878 |</span> +<span class="err">| 3 | 14 | 2.442703 | 2.512545 | 0.727917 | 0.841463 |</span> +<span class="err">| 4 | 15 | 2.442703 | 2.826964 | 0.861380 | 0.853659 |</span> +<span class="err">| 9 | 28 | 2.340435 | 5.492035 | 0.941328 | 0.975610 |</span> +<span class="nt">+-----------+----------+-----------+--------------+-------------------+---------------------+</span> +<span class="nt">Performing</span><span class="na"> feature extraction on resized images...</span> +<span class="na">Completed 64/395</span> +<span class="na">Completed 128/395</span> +<span class="na">Completed 192/395</span> +<span class="na">Completed 256/395</span> +<span class="na">Completed 320/395</span> +<span class="na">Completed 384/395</span> +<span class="na">Completed 395/395</span> +<span class="na">0.9316455696202531</span> +</code></pre></div> + +<p>We just got an accuracy of 94% on Training Data and 97% on Validation Data!</p> + +</main> + + +<script src="assets/manup.min.js"></script> +<script src="/pwabuilder-sw-register.js"></script> +</body> +</html>
\ No newline at end of file |