Note: The input shape for the first layer is 50,50 which corresponds with the sizes of the resized images

Compiling Model

We use the Adam optimiser as it is an adaptive learning rate optimisation algorithm that's been designed specifically for training deep neural networks, which means it changes its learning rate automatically to get the best results

Training Model

We train the model for 10 epochs on the training data and then validate it using the testing data

Results

The cv2. imread() function returns a NumPy array representing the image. Therefore, we need to convert it before we can use it.

Imports

Dataset

Position vs Salary Dataset

We will be using https://drive.google.com/file/d/1tNL4jxZEfpaP4oflfSn6pIHJX7Pachm9/view (Salary vs Position Dataset)

We convert the salary column as the ordinate (y-coordinate) and level column as the abscissa

Defining Stuff

Defining Variables

We first define all the coefficients and constant as tensorflow variables having a random initial value

Model Configuration

Equations

Cost Function

Optimizer

We use the AdamOptimizer for the polynomial functions and GradientDescentOptimizer for the linear function

Model Predictions

Linear Equation

Quadratic Equation

Cubic

Quartic

Quintic

If you do not wish to train on GPU or are running it on your computer, you can ignore the last two lines

Downloading the Dataset

Model Creation

The dataset contains a column named "X1", which is of no use to us. Therefore, we simply drop it

Splitting Dataset

Training

Testing the Model

We have just created our own Fake News Detection Model which has an accuracy of 92%!

Exporting the Model

Note: To download files from Google Colab, simply click on the files section in the sidebar, right click on filename and then click on download

Finally, we implement a simple function which reads the two text fields, creates their bag of words representation and displays an alert with the appropriate result

Converting between image and NumPy array

import numpy
+
+import numpy
 import PIL
 
 # Convert PIL Image to NumPy array
@@ -52,16 +53,19 @@
 
 # Convert array to Image
 img = PIL.Image.fromarray(arr)
-
+

Saving an Image

try:
+
+try:
     img.save(destination, "JPEG", quality=80, optimize=True, progressive=True)
 except IOError:
     PIL.ImageFile.MAXBLOCK = img.size[0] * img.size[1]
     img.save(destination, "JPEG", quality=80, optimize=True, progressive=True)
-
+

diff --git a/docs/posts/2020-01-15-Setting-up-Kaggle-to-use-with-Colab.html b/docs/posts/2020-01-15-Setting-up-Kaggle-to-use-with-Colab.html index 9a7faef..d1c88d4 100644 --- a/docs/posts/2020-01-15-Setting-up-Kaggle-to-use-with-Colab.html +++ b/docs/posts/2020-01-15-Setting-up-Kaggle-to-use-with-Colab.html @@ -69,17 +69,21 @@

Mounting Google Drive

import os
+
+import os
 from google.colab import drive
 drive.mount('/content/drive')
-
+

After this click on the URL in the output section, login and then paste the Auth Code

Configuring Kaggle

os.environ['KAGGLE_CONFIG_DIR'] = "/content/drive/My Drive/"
-

os.environ['KAGGLE_CONFIG_DIR'] = "/content/drive/My Drive/"
+

Voila! You can now download Kaggle datasets

diff --git a/docs/posts/2020-01-16-Image-Classifier-Using-Turicreate.html b/docs/posts/2020-01-16-Image-Classifier-Using-Turicreate.html index 4235b29..5056a82 100644 --- a/docs/posts/2020-01-16-Image-Classifier-Using-Turicreate.html +++ b/docs/posts/2020-01-16-Image-Classifier-Using-Turicreate.html @@ -49,31 +49,40 @@

Mounting Google Drive

import os
+
+import os
 from google.colab import drive
 drive.mount('/content/drive')
-
+

Downloading Dataset from Kaggle

os.environ['KAGGLE_CONFIG_DIR'] = "/content/drive/My Drive/"
+
+os.environ['KAGGLE_CONFIG_DIR'] = "/content/drive/My Drive/"
 !kaggle datasets download ashutosh69/fire-and-smoke-dataset
 !unzip "fire-and-smoke-dataset.zip"
-
+

Pre-Processing

!mkdir default smoke fire
-

!mkdir default smoke fire
+

!ls data/data/img_data/train/default/*.jpg
-

!ls data/data/img_data/train/default/*.jpg
+

img_1002.jpg   img_20.jpg     img_519.jpg     img_604.jpg       img_80.jpg
+
+img_1002.jpg   img_20.jpg     img_519.jpg     img_604.jpg       img_80.jpg
 img_1003.jpg   img_21.jpg     img_51.jpg     img_60.jpg       img_8.jpg
 img_1007.jpg   img_22.jpg     img_520.jpg     img_61.jpg       img_900.jpg
 img_100.jpg    img_23.jpg     img_521.jpg    'img_62 (2).jpg'   img_920.jpg
@@ -106,11 +115,13 @@
 img_204.jpg    img_501.jpg    img_601.jpg     img_78.jpg
 img_205.jpg    img_502.jpg    img_602.jpg     img_79.jpg
 img_206.jpg    img_50.jpg     img_603.jpg     img_7.jpg
-
+

The image files are not actually JPEG, thus we first need to save them in the correct format for Turicreate

from PIL import Image
+
+from PIL import Image
 import glob
 
 
@@ -127,26 +138,32 @@
     rgb_im = im.convert('RGB')
     rgb_im.save((folder + "/" + str(n) + ".jpg"), quality=100)
     n +=1
-
+

!mkdir train
+
+!mkdir train
 !mv default ./train
 !mv smoke ./train
 !mv fire ./train
-
+

Making the Image Classifier

Making an SFrame

!pip install turicreate
-

!pip install turicreate
+

import turicreate as tc
+
+import turicreate as tc
 import os
 
 data = tc.image_analysis.load_images("./train", with_path=True)
@@ -156,11 +173,13 @@
 print(data)
 
 data.save('fire-smoke.sframe')
-
+

+-------------------------+------------------------+
+
++-------------------------+------------------------+
 |           path          |         image          |
 +-------------------------+------------------------+
 |  ./train/default/1.jpg  | Height: 224 Width: 224 |
@@ -194,11 +213,13 @@
 [2028 rows x 3 columns]
 Note: Only the head of the SFrame is printed.
 You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.
-
+

Making the Model

import turicreate as tc
+
+import turicreate as tc
 
 # Load the data
 data =  tc.SFrame('fire-smoke.sframe')
@@ -221,11 +242,13 @@
 
 # Export for use in Core ML
 model.export_coreml('fire-smoke.mlmodel')
-
+

Performing feature extraction on resized images...
+
+Performing feature extraction on resized images...
 Completed   64/1633
 Completed  128/1633
 Completed  192/1633
@@ -283,7 +306,8 @@
 Completed 384/395
 Completed 395/395
 0.9316455696202531
-
+

We just got an accuracy of 94% on Training Data and 97% on Validation Data!

diff --git a/docs/posts/2020-07-01-Install-rdkit-colab.html b/docs/posts/2020-07-01-Install-rdkit-colab.html index 56e2f21..4b5c4e7 100644 --- a/docs/posts/2020-07-01-Install-rdkit-colab.html +++ b/docs/posts/2020-07-01-Install-rdkit-colab.html @@ -55,7 +55,8 @@

Just copy and paste this in a Colab cell and it will install it 👍

import sys
+
+import sys
 import os
 import requests
 import subprocess
@@ -78,7 +79,8 @@
         force=False):
     """install rdkit from miniconda
    
-
+

import rdkit_installer
 rdkit_installer.install()
diff --git a/docs/posts/2020-08-01-Natural-Feature-Tracking-ARJS.html b/docs/posts/2020-08-01-Natural-Feature-Tracking-ARJS.html
index 6b28206..560996e 100644
--- a/docs/posts/2020-08-01-Natural-Feature-Tracking-ARJS.html
+++ b/docs/posts/2020-08-01-Natural-Feature-Tracking-ARJS.html
@@ -184,7 +184,8 @@ me.fset  me.fset3 me.iset
 
 Create a new file called index.html in your project folder. This is the basic template we are going to use. Replace me with the root filename of your image, for example NeverGonnaGiveYouUp.png will become NeverGonnaGiveYouUp. Make sure you have copied all three files from the output folder in the previous step to the root of your project folder.
 
-<script src="https://cdn.jsdelivr.net/gh/aframevr/aframe@1c2407b26c61958baa93967b5412487cd94b290b/dist/aframe-master.min.js"></script>
+
+<script src="https://cdn.jsdelivr.net/gh/aframevr/aframe@1c2407b26c61958baa93967b5412487cd94b290b/dist/aframe-master.min.js"></script>
 <script src="https://raw.githack.com/AR-js-org/AR.js/master/aframe/build/aframe-ar-nft.js"></script>
 
 <style>
@@ -231,7 +232,8 @@ me.fset  me.fset3 me.iset
     <a-entity camera></a-entity>
   </a-scene>
 </body>
-
+
+
 
 In this we are creating a AFrame scene and we are telling it that we want to use NFT Tracking. The amazing part about using AFrame is that we are able to use all AFrame objects!
 
@@ -239,10 +241,12 @@ me.fset  me.fset3 me.iset
 
 Let us add a simple box!
 
-<a-nft .....>
+
+<a-nft .....>
     <a-box position='100 0.5 -180' material='opacity: 0.5; side: double' scale="100 100 100"></a-box>
 </a-nft>
-
+
+
 
 Now to test it out we will need to create a simple server, I use Python's inbuilt SimpleHTTPServer alongside ngrok 
 
@@ -277,12 +281,14 @@ Serving HTTP on 0.0.0.0 port 8000 ...
 
 Edit your index.html 
 
-<a-nft ..>
+
+<a-nft ..>
     <a-box ..>
         <a-torus-knot radius='0.26' radius-tubular='0.05' ></a-torus-knot>
     </ a-box>
 </ a-nft>
-
+
+
 
 
 
@@ -298,9 +304,11 @@ Serving HTTP on 0.0.0.0 port 8000 ...
 
 Change the box's material to add the GIF shader
 
-...
+
+...
 <a-box position='100 0.5 -180' material="shader:gif;src:url(https://media.tenor.com/images/412b1aa9149d98d561df62db221e0789/tenor.gif);opacity:.5" .....>
-
+
+
 
 
 
diff --git a/docs/posts/2020-10-11-macOS-Virtual-Cam-OBS.html b/docs/posts/2020-10-11-macOS-Virtual-Cam-OBS.html
index 06951dc..d99f7b8 100644
--- a/docs/posts/2020-10-11-macOS-Virtual-Cam-OBS.html
+++ b/docs/posts/2020-10-11-macOS-Virtual-Cam-OBS.html
@@ -43,7 +43,7 @@
 
 	Trying Different Camera Setups
 
-
+
 Animated Overlays
 Using a modern camera as your webcam
 Using your phone's camera as your webcam
diff --git a/docs/posts/2020-11-17-Lets-Encrypt-DuckDns.html b/docs/posts/2020-11-17-Lets-Encrypt-DuckDns.html
index f8e7b6c..fdde2b8 100644
--- a/docs/posts/2020-11-17-Lets-Encrypt-DuckDns.html
+++ b/docs/posts/2020-11-17-Lets-Encrypt-DuckDns.html
@@ -47,13 +47,17 @@
 
 Dependencies
 
-sudo apt update && sudo apt install certbot -y
-
+
+sudo apt update && sudo apt install certbot -y
+
+
 
 Get the Certificate
 
-sudo certbot certonly --manual --preferred-challenges dns-01 --email senpai@email.com -d mydomain.duckdns.org
-
+
+sudo certbot certonly --manual --preferred-challenges dns-01 --email senpai@email.com -d mydomain.duckdns.org
+
+
 
 After you accept that you are okay with you IP address being logged, it will prompt you with updating your dns record. You need to create a new TXT record in the DNS settings for your domain.
 
@@ -66,7 +70,8 @@
 
 You can check if the TXT records have been updated by using the dig command:
 
-dig navanspi.duckdns.org TXT
+
+dig navanspi.duckdns.org TXT
 ; <<>> DiG 9.16.1-Ubuntu <<>> navanspi.duckdns.org TXT
 ;; global options: +cmd
 ;; Got answer:
@@ -85,7 +90,8 @@ navanspi.duckdns.org.    60    IN    TXT    ;; SERVER: 127.0.0.53#53(127.0.0.53)
 ;; WHEN: Tue Nov 17 15:23:15 IST 2020
 ;; MSG SIZE  rcvd: 105
-
+
+
 
 DuckDNS almost instantly propagates the changes but for other domain hosts, it could take a while. 
 
@@ -99,13 +105,17 @@ navanspi.duckdns.org.    60    IN    TXT    gunicorn api:app -k uvicorn.workers.UvicornWorker -b 0.0.0.0:7589
-
+
+gunicorn api:app -k uvicorn.workers.UvicornWorker -b 0.0.0.0:7589
+
+
 
 To use the certificate with it, simply copy the cert.pem and privkey.pem to your working directory ( change the appropriate permissions ) and include them in the command
 
-gunicorn api:app -k uvicorn.workers.UvicornWorker -b 0.0.0.0:7589 --certfile=cert.pem --keyfile=privkey.pem
-
+
+gunicorn api:app -k uvicorn.workers.UvicornWorker -b 0.0.0.0:7589 --certfile=cert.pem --keyfile=privkey.pem
+
+
 
 Caveats with copying the certificate: If you renew the certificate you will have to re-copy the files
 
diff --git a/docs/posts/2020-12-1-HTML-JS-RSS-Feed.html b/docs/posts/2020-12-1-HTML-JS-RSS-Feed.html
index 4fdb015..8acc446 100644
--- a/docs/posts/2020-12-1-HTML-JS-RSS-Feed.html
+++ b/docs/posts/2020-12-1-HTML-JS-RSS-Feed.html
@@ -45,7 +45,8 @@
 
 If you want to directly open the HTML file in your browser after saving, don't forget to set CORS_PROXY="" 
 
-<!doctype html>
+
+<!doctype html>
 <html lang="en">
 <head>
   <meta charset="utf-8">
@@ -240,7 +241,8 @@
 </script>
 <noscript>Uh Oh! Your browser does not support JavaScript or JavaScript is currently disabled. Please enable JavaScript or switch to a different browser.</noscript>
 </body></html>
-
+
+
 
 	
diff --git a/docs/posts/2021-06-25-Blog2Twitter-P1.html b/docs/posts/2021-06-25-Blog2Twitter-P1.html
index ada9666..62233ab 100644
--- a/docs/posts/2021-06-25-Blog2Twitter-P1.html
+++ b/docs/posts/2021-06-25-Blog2Twitter-P1.html
@@ -57,7 +57,8 @@ I am not handling lists or images right now.
 
 pip install tweepy
 
-import os
+
+import os
 import tweepy
 
 consumer_key = os.environ["consumer_key"]
@@ -70,13 +71,15 @@ I am not handling lists or images right now.
 auth.set_access_token(access_token, access_token_secret)
 
 api = tweepy.API(auth)
-
+
+
 
 The program need to convert the blog post into text fragments.
 
 It reads the markdown file, removes the top YAML content, checks for headers and splits the content.
 
-tweets = []
+
+tweets = []
 
 first___n = 0
 
@@ -103,13 +106,15 @@ I am not handling lists or images right now.
                 print("ERROR")
             else:
                 tweets.append(line)
-
+
+
 
 Every status update using tweepy has an id attached to it, for the next tweet in the thread, it adds that ID while calling the function.
 
 For every tweet fragment, it also appends 1/n.
 
-for idx, tweet in enumerate(tweets):
+
+for idx, tweet in enumerate(tweets):
     tweet += " {}/{}".format(idx+1,len(tweets))
     if idx == 0:
         a = None
@@ -118,12 +123,15 @@ I am not handling lists or images right now.
         a = api.update_status(tweet,in_reply_to_status_id=a.id)
     print(len(tweet),end=" ")
     print("{}/{}\n".format(idx+1,len(tweets)))
-
+
+
 
 Finally, it replies to the last tweet in the thread with the link of the post.
 
-api.update_status("Web Version: {}".format(post_link))
-
+
+api.update_status("Web Version: {}".format(post_link))
+
+
 
 Result
 
diff --git a/docs/posts/2021-06-27-Crude-ML-AI-Powered-Chatbot-Swift.html b/docs/posts/2021-06-27-Crude-ML-AI-Powered-Chatbot-Swift.html
index 0b307fd..cdae911 100644
--- a/docs/posts/2021-06-27-Crude-ML-AI-Powered-Chatbot-Swift.html
+++ b/docs/posts/2021-06-27-Crude-ML-AI-Powered-Chatbot-Swift.html
@@ -89,7 +89,8 @@ I created a sample JSON with only 3 examples (I know, very less, but works for a
 
 
 
-[
+
+[
     {
         "tokens": ["Tell","me","about","the","drug","Aspirin","."],
         "labels": ["NONE","NONE","NONE","NONE","NONE","COMPOUND","NONE"]
@@ -103,7 +104,8 @@ I created a sample JSON with only 3 examples (I know, very less, but works for a
         "labels": ["NONE","NONE","NONE","NONE","COMPOUND","NONE","NONE"]
     }
 ]
-
+
+
 
 
 
@@ -113,7 +115,8 @@ I created a sample JSON with only 3 examples (I know, very less, but works for a
 
 
 
-import CoreML
+
+import CoreML
 import NaturalLanguage
 
 let mlModelClassifier = try IntentDetection_1(configuration:  MLModelConfiguration()).model
@@ -124,7 +127,8 @@ I created a sample JSON with only 3 examples (I know, very less, but works for a
 
 let tagger = NLTagger(tagSchemes: [.nameType, NLTagScheme("Apple")])
 tagger.setModels([tagPredictor], forTagScheme: NLTagScheme("Apple"))
-
+
+
 
 Now, we define a simple structure which the custom function(s) can use to access the provided input. 
 It can also be used to hold additional variables.
@@ -134,7 +138,8 @@ The latter can be replaced with a function which asks the user for the input. 
 
-struct User {
+
+struct User {
     static var message = ""
 }
 
@@ -158,14 +163,16 @@ The latter can be replaced with a function which asks the user for the input. }
 
 }
-
+
+
 
 Sometimes, no action needs to be performed, and the bot can use a predefined set of responses. 
 Otherwise, if an action is required, it can call the custom action.
 
 
 
-let defaultResponses = [
+
+let defaultResponses = [
     "greetings": "Hello",
     "banter": "no, plix no"
 ]
@@ -173,14 +180,16 @@ Otherwise, if an action is required, it can call the custom action.
 let customActions = [
     "deez-drug": customAction
 ]
-
+
+
 
 In the sample input, the program is updating the User.message and checking if it has a default response.
 Otherwise, it calls the custom action.
 
 
 
-let sampleMessages = [
+
+let sampleMessages = [
     "Hey there, how is it going",
     "hello, there",
     "Who let the dogs out",
@@ -200,7 +209,8 @@ Otherwise, it calls the custom action.
         print(customActions[prediction!]!())
     }
 }
-
+
+
 
 
 
diff --git a/docs/posts/2022-05-21-Similar-Movies-Recommender.html b/docs/posts/2022-05-21-Similar-Movies-Recommender.html
index 5d2d6fe..f45b45e 100644
--- a/docs/posts/2022-05-21-Similar-Movies-Recommender.html
+++ b/docs/posts/2022-05-21-Similar-Movies-Recommender.html
@@ -63,7 +63,8 @@
 
 First, I needed to check the total number of records in Trakt’s database.
 
-import requests
+
+import requests
 import os
 
 trakt_id = os.getenv("TRAKT_ID")
@@ -87,14 +88,16 @@
 res = requests.get(f"{api_base}/search/movie",headers=headers,params=params)
 total_items = res.headers["x-pagination-item-count"]
 print(f"There are {total_items} movies")
-
+
+
 
 There are 333946 movies
 
 
 First, I needed to declare the database schema in (database.py):
 
-import sqlalchemy
+
+import sqlalchemy
 from sqlalchemy import create_engine
 from sqlalchemy import Table, Column, Integer, String, MetaData, ForeignKey, PickleType
 from sqlalchemy import insert
@@ -129,13 +132,15 @@
     meta.create_all(engine)
     Session = sessionmaker(bind=engine)
     return engine, Session
-
+
+
 
 In the end, I could have dropped the embeddings field from the table schema as I never got around to using it.
 
 Scripting Time
 
-from database import *
+
+from database import *
 from tqdm import tqdm
 import requests
 import os
@@ -228,7 +233,8 @@
                 except IntegrityError:
                     trans.rollback()
     req_count += 1
-
+
+
 
 (Note: I was well within the rate-limit so I did not have to slow down or implement any other measures)
 
@@ -263,7 +269,8 @@ As of writing this post, I did not include any other database except Trakt. 
 Installing the Python module (pinecone-client)

A new method to blog

+ +

Paper Website is a service that lets you build a website with just pen and paper. I am going to try and replicate the process.

+ +

The Plan

+ +

The continuity feature on macOS + iOS lets you scan PDFs directly from your iPhone. I want to be able to scan these pages and automatically run an Automator script that takes the PDF and OCRs the text. Then I can further clean the text and convert from markdown.

+ +

Challenges

+ +

I quickly realised that the OCR software I planned on using could not detect my shitty handwriting accurately. I tried using ABBY Finereader, Prizmo and OCRMyPDF. (Abby Finereader and Prizmo support being automated by Automator).

+ +

Now, I could either write neater, or use an external API like Microsoft Azure

+ +

Solution

+ +

OCR

+ +

In the PDFs, all the scans are saved as images on a page. I extract the image and then send it to Azure's API.

+ +

Paragraph Breaks

+ +

The recognised text had multiple lines breaking in the middle of the sentence, Therefore, I use what is called a pilcrow to specify paragraph breaks. But, rather than trying to draw the normal pilcrow, I just use the HTML entity ¶ which is the pilcrow character.

+ +

Where is the code?

+ +

I created a GitHub Gist for a sample Python script to take the PDF and print the text

+ +

A more complete version with Auomator scripts and an entire publishing pipeline will be available as a GitHub and Gitea repo soon.

+ +

* In Part 2, I will discuss some more features *

+ + +

+ +

Result

Mounting Google Drive

Configuring Kaggle

Imports

Dataset

Fetching the Data

Processing the Data

Splitting Data

Compiling Model

Training Model

Results

Scripting Time

Testing it Out

The Plan

Challenges

Solution

OCR

Paragraph Breaks

Where is the code?

Imports

Dataset

Position vs Salary Dataset

Defining Stuff

Defining Variables

Model Configuration

Equations

Cost Function

Optimizer

Model Predictions

Linear Equation

Quadratic Equation

Cubic

Quartic

Quintic

Dependencies

Get the Certificate

Downloading the Dataset

Model Creation

Splitting Dataset

Training

Testing the Model

Exporting the Model

Saving an Image

Mounting Google Drive

Downloading Dataset from Kaggle

Pre-Processing

Making the Image Classifier

Making an SFrame

Making the Model

Imports

Dataset

Fetching the Data

Processing the Data

Splitting Data

Compiling Model

Training Model

Results

Imports

Dataset

Position vs Salary Dataset

Defining Stuff

Defining Variables

Model Configuration

Equations

Cost Function

Optimizer

Model Predictions

Linear Equation

Quadratic Equation

Cubic

Quartic

Quintic

Downloading the Dataset

Model Creation

Splitting Dataset

Training

Testing the Model

Exporting the Model

Converting between image and NumPy array

Saving an Image