diff options
author | Navan Chauhan <navanchauhan@gmail.com> | 2024-04-03 09:36:58 -0600 |
---|---|---|
committer | GitHub <noreply@github.com> | 2024-04-03 09:36:58 -0600 |
commit | 84ad5d8360fb0683f12f9bae8f29d6b36964dde3 (patch) | |
tree | 64bb1d9675ac3d6a3604d512311a754b08c53be2 /Content/posts | |
parent | 0e4b5fcc164cd8432678bdffd58b8c123acee186 (diff) | |
parent | d00b2da99ef2d498074a6e4eb8d80995cdfc4476 (diff) |
Merge branch 'vanilla-ssg' into dependabot/pip/jinja2-3.1.3
Diffstat (limited to 'Content/posts')
7 files changed, 1042 insertions, 0 deletions
diff --git a/Content/posts/2020-07-01-Install-rdkit-colab.md b/Content/posts/2020-07-01-Install-rdkit-colab.md index ff5da1e..4a87bf1 100644 --- a/Content/posts/2020-07-01-Install-rdkit-colab.md +++ b/Content/posts/2020-07-01-Install-rdkit-colab.md @@ -6,6 +6,16 @@ tags: Tutorial, Code-Snippet, Colab # Installing RDKit on Google Colab +**Update: March 2024** + +`rdkit-pypi` has been deprecated in favour of `rdkit` + +You can simply run: + +``` +!pip install rdkit +``` + **EDIT:** Try installing RDKit using pip ``` diff --git a/Content/posts/2024-02-26-control-element-under-another-element-html-css.md b/Content/posts/2024-02-26-control-element-under-another-element-html-css.md new file mode 100644 index 0000000..cbdfe6a --- /dev/null +++ b/Content/posts/2024-02-26-control-element-under-another-element-html-css.md @@ -0,0 +1,77 @@ +--- +date: 2024-02-26 11:57 +description: With CSS you can disable any interactions with an element and directly control the underlying element +tags: HTML, CSS +draft: false +--- + +# Interacting with underlying element in HTML + +I know that the title is a bit weird. I was trying to interact with a video under an iPhone Bezel Screen frame. + +```html +<div class="row-span-2 md:col-span-1 rounded-xl border-2 border-slate-400/10 bg-neutral-100 p-4 dark:bg-neutral-900"> + <div class="content flex flex-wrap content-center justify-center"> + <img src="iphone-12-white.png" class="h-[60vh] z-10 absolute"> + <!--<img src="screenshot2.jpeg" class="h-[57vh] mt-4 mr-1 rounded-[2rem]">--> + <video src="screenrec.mp4" class="h-[57vh] mt-4 mr-1 rounded-[2rem]" controls muted autoplay></video> + </div> +</div> +``` + +![Video Under a Transparent Image](/assets/underlying/video-under-element.jpg) + +Turns out, you can disable pointer events! + +In Tailwind, it is as simple as adding `pointer-events-none` to the bezel screen. + +In CSS, this can be done by: + +```css +.className { + pointer-events: none +} +``` + +Let us try this in a simple example. + +## Example + +Here, we create a button and overlay a transparent box + +```html +<div style="height: 200px; width: 300px; background-color: rgba(255, 0, 0, 0.4); z-index: 2; position: absolute;"> +A box with 200px height and 200px width +</div> +<button style="z-index: 1; margin-top: 20px; margin-bottom: 200px;" onclick="alert('You were able to click this button')">Try clicking me</button> +``` + +<hr> + +<div style="height: 200px; width: 300px; background-color: rgba(255, 0, 0, 0.4); z-index: 2; position: absolute;"> +A box with 200px height and 300px width +</div> +<button style="z-index: 1; margin-top: 20px; margin-bottom: 200px;" onclick="alert('You were able to click this button')">Try clicking me</button> +<hr> + +As you can see, you cannot click the button because the red box comes in the way. We can fix this by adding `pointer-events: none` to the box. + +```html +<div style="height: 200px; width: 300px; background-color: rgba(0, 255, 0, 0.4); z-index: 2; position: absolute; pointer-events: none;"> +A box with 200px height and 300px width +</div> +<button style="z-index: 1; margin-top: 20px; margin-bottom: 200px" onclick="alert('You were able to click this button')">Try clicking me</button> +</div> +``` + +<hr> + +<div style="height: 200px; width: 300px; background-color: rgba(0, 255, 0, 0.4); z-index: 2; position: absolute; pointer-events: none;"> +A box with 200px height and 300px width +</div> +<button style="z-index: 1; margin-top: 20px; margin-bottom: 200px" onclick="alert('You were able to click this button')">Try clicking me</button> +</div> + + + + diff --git a/Content/posts/2024-03-04-opencvjs-video-processing.md b/Content/posts/2024-03-04-opencvjs-video-processing.md new file mode 100644 index 0000000..cf87fb1 --- /dev/null +++ b/Content/posts/2024-03-04-opencvjs-video-processing.md @@ -0,0 +1,12 @@ +--- +date: 2024-03-04 12:01 +description: Real time video processing with OpenCV.js in the browser +tags: HTML, JS, OpenCV +draft: true +--- + +# Video Processing W/ OpenCV.js + +Note: A really weird bug you might run into when you are trying to mess around +and reloading the page on iOS is that OpenCV just stops loading. This is a bug in +WebKit.: diff --git a/Content/posts/2024-03-15-setting-up-macos-for-8088-dos-dev.md b/Content/posts/2024-03-15-setting-up-macos-for-8088-dos-dev.md new file mode 100644 index 0000000..15161f5 --- /dev/null +++ b/Content/posts/2024-03-15-setting-up-macos-for-8088-dos-dev.md @@ -0,0 +1,267 @@ +--- +date: 2024-03-15 13:16 +description: This goes through compiling Open Watcom 2 and creating simple hello-world exampls +tags: DOS, x86, macOS +draft: false +--- + +# Cross-Compiling Hello World for DOS on macOS + +Technically this should work for any platform that OpenWatcom 2 supports compiling binaries for. Some instructions are based on [a post at retrocoding.net](https://retrocoding.net/openwatcom-gateway-to-ancient-world-of-x86), + and [John Tsiombikas's post](http://nuclear.mutantstargoat.com/articles/retrocoding/dos01-setup/#hello-world-program) + +## Prerequisites + +You should already have XCode / Command Line Tools, and Homebrew installed. To compile Open Watcom for DOS you will need DOSBox (I use DOSBox-X). + +```bash +brew install --cask dosbox-x +``` + +## Compiling Open Watcom v2 + +*If this process is super annoying, I might make a custom homebrew tap to build and install Open Watcom* + +```bash +git clone https://github.com/open-watcom/open-watcom-v2 +cp open-watcom-v2/setvars.sh custom_setvars.sh +``` + +Now, edit this `setvars.sh` file. My file looks like this: + +```bash +#!/bin/zsh +export OWROOT="/Users/navanchauhan/Developer/8088Stuff/open-watcom-v2" +export OWTOOLS=CLANG +export OWDOCBUILD=0 +export OWGUINOBUILD=0 +export OWDISTRBUILD=0 +export OWDOSBOX="/Applications/dosbox-x.app/Contents/MacOS/dosbox-x" +export OWOBJDIR=binbuildV01 +. "$OWROOT/cmnvars.sh" +echo "OWROOT=$OWROOT" +cd "$OWROOT" +``` + +Note, your `OWRTOOT` is definitely going to be in a different location. + +```bash +source ./custom_setvars.sh +./build.sh +./build.sh rel +``` + +This will build, and then copy everything to the `rel` directory inside `open-watcom-v2` directory. Since I ran this on an Apple Silicon Mac, + all the binaries for me are in the `armo64` directory. You can now move everything inside the rel folder to another location, or create a simple + script to init all variables whenever you want. + +I like having a script called `exportVarsForDOS.sh` + +```bash +#!/bin/zsh + +export WATCOM=/Users/navanchauhan/Developer/8088Stuff/open-watcom-v2/rel +export PATH=$PATH:$WATCOM/armo64 +export EDDAT=$WATCOM/eddat + +# For DOS 8088/8086 development +export INCLUDE=$WATCOM/h +export LIB=$WATCOM/lib286 # You don't really need this +``` + +Then, when you need to load up these variables, you can simply run `source exportVarsForDOS.sh` or `. exportVarsForDOS.sh` + +## Hello World + +### Buliding without any Makefiles + +Create a new file called `example1.c` + +```c +#include<stdio.h> + +int main() { + printf("Hello World!"); + return 0; +} +``` + +First we compile the code: + +```bash +$ wcc example1.c +Open Watcom C x86 16-bit Optimizing Compiler +Version 2.0 beta Mar 15 2024 13:11:55 +Copyright (c) 2002-2024 The Open Watcom Contributors. All Rights Reserved. +Portions Copyright (c) 1984-2002 Sybase, Inc. All Rights Reserved. +Source code is available under the Sybase Open Watcom Public License. +See https://github.com/open-watcom/open-watcom-v2#readme for details. +example1.c: 7 lines, included 818, 0 warnings, 0 errors +Code size: 19 +``` + +Then, link to make an executable: + +```bash +$ wlink name example1.exe system dos file example1.o +Open Watcom Linker Version 2.0 beta Mar 15 2024 13:10:09 +Copyright (c) 2002-2024 The Open Watcom Contributors. All Rights Reserved. +Portions Copyright (c) 1985-2002 Sybase, Inc. All Rights Reserved. +Source code is available under the Sybase Open Watcom Public License. +See https://github.com/open-watcom/open-watcom-v2#readme for details. +loading object files +searching libraries +creating a DOS executable +``` + +If you want to test this executable, jump to the section titled `Testing with DOSBox-X` below. + +### Simple Makefile + +```makefile +obj = main.o hello.o +bin = tizts.com + +CC = wcc +CFLAGS = -0 +LD = wlink + +$(bin): $(obj) + $(LD) name $@ system dos file main.o file hello.o + +.c.o: + $(CC) $(CFLAGS) $< + +clean: + rm $(obj) $(bin) +``` + +Where, `main.c` +```c +void hello(void); + +int main(void) +{ + hello(); + return 0; +} +``` + +and `hello.c` + +```c +/* hello.c */ +#include <stdio.h> + +void hello(void) +{ + printf("Hello!"); +} +``` + +To compile into `tizts.com` simply run `wmake` + +```bash +$ wmake +➜ simple-cpp wmake +Open Watcom Make Version 2.0 beta Mar 15 2024 13:10:16 +Copyright (c) 2002-2024 The Open Watcom Contributors. All Rights Reserved. +Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved. +Source code is available under the Sybase Open Watcom Public License. +See https://github.com/open-watcom/open-watcom-v2#readme for details. + wcc -0 main.c +Open Watcom C x86 16-bit Optimizing Compiler +Version 2.0 beta Mar 15 2024 13:11:55 +Copyright (c) 2002-2024 The Open Watcom Contributors. All Rights Reserved. +Portions Copyright (c) 1984-2002 Sybase, Inc. All Rights Reserved. +Source code is available under the Sybase Open Watcom Public License. +See https://github.com/open-watcom/open-watcom-v2#readme for details. +main.c(8): Warning! W138: No newline at end of file +main.c: 8 lines, included 53, 1 warnings, 0 errors +Code size: 12 + wcc -0 hello.c +Open Watcom C x86 16-bit Optimizing Compiler +Version 2.0 beta Mar 15 2024 13:11:55 +Copyright (c) 2002-2024 The Open Watcom Contributors. All Rights Reserved. +Portions Copyright (c) 1984-2002 Sybase, Inc. All Rights Reserved. +Source code is available under the Sybase Open Watcom Public License. +See https://github.com/open-watcom/open-watcom-v2#readme for details. +hello.c: 8 lines, included 818, 0 warnings, 0 errors +Code size: 17 + wlink name tizts.com system dos file main.o file hello.o +Open Watcom Linker Version 2.0 beta Mar 15 2024 13:10:09 +Copyright (c) 2002-2024 The Open Watcom Contributors. All Rights Reserved. +Portions Copyright (c) 1985-2002 Sybase, Inc. All Rights Reserved. +Source code is available under the Sybase Open Watcom Public License. +See https://github.com/open-watcom/open-watcom-v2#readme for details. +loading object files +searching libraries +creating a DOS executable +``` + +### Using CMake + +Create a file called `CMakeLists.txt` + +```CMake +project(hello) + +set(SOURCES abc.c) + +add_executable(hello ${SOURCES}) +``` + +Where, `abc.c` is: + +```c +#include <stdio.h> + +int main() { + printf("Does this work?"); + return 0; +} +``` + +```bash +mkdir build +cd build +``` + +And build using CMake + +``` +cmake -DCMAKE_SYSTEM_NAME=DOS -DCMAKE_SYSTEM_PROCESSOR=I86 -DCMAKE_C_FLAGS="-0 -bt=dos -d0 -oaxt" -G "Watcom WMake" ../.. +``` + +There you have it. Three different ways to compile a C program on a macOS device in 2024 that can run on an IBM PC 5150 (which was released in 1981!) + +## Testing with DOSBox-X + +```bash +cp example1.exe ~/Downloads +/Applications/dosbox-x.app/Contents/MacOS/dosbox-x +``` + +In DOSBox-X we now mount the `~/Downloads` folder as our `C:` drive + +``` +mount C ~/Downloads +``` + +Switch to the C drive + +``` +C: +``` + +Run the program: + +``` +example1 +``` + +![Running our program in DOSBox-X](/assets/posts/dosbox/hello-world.png) + +*My DOSBox setup might look slightly different than yours...* + + diff --git a/Content/posts/2024-03-21-Polynomial-Regression-in-TensorFlow-2.md b/Content/posts/2024-03-21-Polynomial-Regression-in-TensorFlow-2.md new file mode 100644 index 0000000..6317175 --- /dev/null +++ b/Content/posts/2024-03-21-Polynomial-Regression-in-TensorFlow-2.md @@ -0,0 +1,242 @@ +--- +date: 2024-03-21 12:46 +description: Predicting n-th degree polynomials using TensorFlow 2.x +tags: Tutorial, Tensorflow, Colab +--- + +# Polynomial Regression Using TensorFlow 2.x + +I have a similar post titled [Polynomial Regression Using Tensorflow](/posts/2019-12-16-TensorFlow-Polynomial-Regression.html) that used `tensorflow.compat.v1` (Which still works as of TF 2.16). But, I thought it would be nicer to redo it with newer TF versions. + +I will be skipping all the introductions about polynomial regression and jumping straight to the code. Personally, I prefer using `scikit-learn` for this task. + +## Position vs Salary Dataset + +Again, we will be using https://drive.google.com/file/d/1tNL4jxZEfpaP4oflfSn6pIHJX7Pachm9/view (Salary vs Position Dataset) + +If you are in a Python Notebook environment like Kaggle or Google Colaboratory, you can simply run: +```Termcap +!wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=1tNL4jxZEfpaP4oflfSn6pIHJX7Pachm9' -O data.csv +``` + +## Code + +If you just want to copy-paste the code, scroll to the bottom for the entire snippet. Here I will try and walk through setting up code for a 3rd-degree (cubic) polynomial + +### Imports + +```python +import pandas as pd +import tensorflow as tf +import matplotlib.pyplot as plt +import numpy as np +``` + +### Reading the Dataset + +```python +df = pd.read_csv("data.csv") +``` + +### Variables and Constants + +Here, we initialize the X and Y values as constants, since they are not going to change. The coefficients are defined as variables. + +```python +X = tf.constant(df["Level"], dtype=tf.float32) +Y = tf.constant(df["Salary"], dtype=tf.float32) + +coefficients = [tf.Variable(np.random.randn() * 0.01, dtype=tf.float32) for _ in range(4)] +``` + +Here, `X` and `Y` are the values from our dataset. We initialize the coefficients for the equations as small random values. + +These coefficients are evaluated by Tensorflow's `tf.math.poyval` function which returns the n-th order polynomial based on how many coefficients are passed. Since our list of coefficients contains 4 different variables, it will be evaluated as: + +``` +y = (x**3)*coefficients[3] + (x**2)*coefficients[2] + (x**1)*coefficients[1] (x**0)*coefficients[0] +``` + +Which is equivalent to the general cubic equation: + +<script src="https://cdn.jsdelivr.net/npm/mathjax@4.0.0-beta.4/tex-mml-chtml.js" id="MathJax-script"></script> +<script src="https://cdn.jsdelivr.net/npm/mathjax@4.0.0-beta.4/input/tex/extensions/noerrors.js" charset="UTF-8"></script> + +$$ +y = ax^3 + bx^2 + cx + d +$$ + +### Optimizer Selection & Training + +```python +optimizer = tf.keras.optimizers.Adam(learning_rate=0.3) +num_epochs = 10_000 + +for epoch in range(num_epochs): + with tf.GradientTape() as tape: + y_pred = tf.math.polyval(coefficients, X) + loss = tf.reduce_mean(tf.square(y - y_pred)) + grads = tape.gradient(loss, coefficients) + optimizer.apply_gradients(zip(grads, coefficients)) + if (epoch+1) % 1000 == 0: + print(f"Epoch: {epoch+1}, Loss: {loss.numpy()}" +``` + +In TensorFlow 1, we would have been using `tf.Session` instead. + +Here we are using `GradientTape()` instead, to keep track of the loss evaluation and coefficients. This is crucial, as our optimizer needs these gradients to be able to optimize our coefficients. + +Our loss function is Mean Squared Error (MSE): + +$$ += \frac{1}{n} \sum_{i=1}^{n}{(Y\_i - \hat{Y\_i})^2} +$$ + +Where <math xmlns="http://www.w3.org/1998/Math/MathML"><mover><msub><mi>Y</mi><mi>i</mi></msub><mo stretchy="false" style="math-style:normal;math-depth:0;">^</mo></mover></math> is the predicted value and <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>Y</mi><mi>i</mi></msub></math> is the actual value + +### Plotting Final Coefficients + +```python +final_coefficients = [c.numpy() for c in coefficients] +print("Final Coefficients:", final_coefficients) + +plt.plot(df["Level"], df["Salary"], label="Original Data") +plt.plot(df["Level"],[tf.math.polyval(final_coefficients, tf.constant(x, dtype=tf.float32)).numpy() for x in df["Level"]]) +plt.ylabel('Salary') +plt.xlabel('Position') +plt.title("Salary vs Position") +plt.show() +``` + + +## Code Snippet for a Polynomial of Degree N + +### Using Gradient Tape + +This should work regardless of the Keras backend version (2 or 3) + +```python +import tensorflow as tf +import numpy as np +import pandas as pd +import matplotlib.pyplot as plt + +df = pd.read_csv("data.csv") + +############################ +## Change Parameters Here ## +############################ +x_column = "Level" # +y_column = "Salary" # +degree = 2 # +learning_rate = 0.3 # +num_epochs = 25_000 # +############################ + +X = tf.constant(df[x_column], dtype=tf.float32) +Y = tf.constant(df[y_column], dtype=tf.float32) + +coefficients = [tf.Variable(np.random.randn() * 0.01, dtype=tf.float32) for _ in range(degree + 1)] + +optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate) + +for epoch in range(num_epochs): + with tf.GradientTape() as tape: + y_pred = tf.math.polyval(coefficients, X) + loss = tf.reduce_mean(tf.square(Y - y_pred)) + grads = tape.gradient(loss, coefficients) + optimizer.apply_gradients(zip(grads, coefficients)) + if (epoch+1) % 1000 == 0: + print(f"Epoch: {epoch+1}, Loss: {loss.numpy()}") + +final_coefficients = [c.numpy() for c in coefficients] +print("Final Coefficients:", final_coefficients) + +print("Final Equation:", end=" ") +for i in range(degree+1): + print(f"{final_coefficients[i]} * x^{degree-i}", end=" + " if i < degree else "\n") + +plt.plot(X, Y, label="Original Data") +plt.plot(X,[tf.math.polyval(final_coefficients, tf.constant(x, dtype=tf.float32)).numpy() for x in df[x_column]]), label="Our Poynomial" +plt.ylabel(y_column) +plt.xlabel(x_column) +plt.title(f"{x_column} vs {y_column}") +plt.legend() +plt.show() +``` + +### Without Gradient Tape + +This relies on the Optimizer's `minimize` function and uses the `var_list` parameter to update the variables. + +This will not work with Keras 3 backend in TF 2.16.0 and above unless you switch to the legacy backend. + +```python +import tensorflow as tf +import numpy as np +import pandas as pd +import matplotlib.pyplot as plt + +df = pd.read_csv("data.csv") + +############################ +## Change Parameters Here ## +############################ +x_column = "Level" # +y_column = "Salary" # +degree = 2 # +learning_rate = 0.3 # +num_epochs = 25_000 # +############################ + +X = tf.constant(df[x_column], dtype=tf.float32) +Y = tf.constant(df[y_column], dtype=tf.float32) + +coefficients = [tf.Variable(np.random.randn() * 0.01, dtype=tf.float32) for _ in range(degree + 1)] + +optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate) + +def loss_function(): + pred_y = tf.math.polyval(coefficients, X) + return tf.reduce_mean(tf.square(pred_y - Y)) + +for epoch in range(num_epochs): + optimizer.minimize(loss_function, var_list=coefficients) + if (epoch+1) % 1000 == 0: + current_loss = loss_function().numpy() + print(f"Epoch {epoch+1}: Training Loss: {current_loss}") + +final_coefficients = coefficients.numpy() +print("Final Coefficients:", final_coefficients) + +print("Final Equation:", end=" ") +for i in range(degree+1): + print(f"{final_coefficients[i]} * x^{degree-i}", end=" + " if i < degree else "\n") + +plt.plot(X, Y, label="Original Data") +plt.plot(X,[tf.math.polyval(final_coefficients, tf.constant(x, dtype=tf.float32)).numpy() for x in df[x_column]], label="Our Polynomial") +plt.ylabel(y_column) +plt.xlabel(x_column) +plt.legend() +plt.title(f"{x_column} vs {y_column}") +plt.show() +``` + + +As always, remember to tweak the parameters and choose the correct model for the job. A polynomial regression model might not even be the best model for this particular dataset. + +## Further Programming + +How would you modify this code to use another type of nonlinear regression? Say, + +$$ y = ab^x $$ + +Hint: Your loss calculation would be similar to: + +```python +bx = tf.pow(coefficients[1], X) +pred_y = tf.math.multiply(coefficients[0], bx) +loss = tf.reduce_mean(tf.square(pred_y - Y)) +``` + + diff --git a/Content/posts/2024-03-26-Derivation-of-the-Quadratic-Equation.md b/Content/posts/2024-03-26-Derivation-of-the-Quadratic-Equation.md new file mode 100644 index 0000000..0435a6c --- /dev/null +++ b/Content/posts/2024-03-26-Derivation-of-the-Quadratic-Equation.md @@ -0,0 +1,55 @@ +--- +date: 2024-03-26 15:36 +description: Quick derivation of the quadratic equation by completing the square +tags: mathematics +--- + +# Quadratic Formula Derivation + +The standard form of a quadratic equation is: + +$$ +ax^2 + bx + c = 0 +$$ + +Here, $a, b, c \in \mathbb{R}$, and $a \neq 0$ + +We begin by first dividing both sides by the coefficient $a$ + +$$ +\implies x^2 + \frac{b}{a}x + \frac{c}{a} = 0 +$$ + +We can rearrange the equation: + +$$ +x^2 + \frac{b}{a}x = - \frac{c}{a} +$$ + +We can then use the method of completing the square. ([Maths is Fun](https://www.mathsisfun.com/algebra/completing-square.html) has a really good explanation for this technique) + +$$ +x^2 + \frac{b}{a}x + (\frac{b}{2a})^2 = \frac{-c}{a} + (\frac{b}{2a})^2 +$$ + +On our LHS, we can clearly recognize that it is the expanded form of $(x + d)^2$ i.e $x^2 + 2x\cdot d + d^2$ + +$$ +\implies (x + \frac{b}{2a})^2 = \frac{-c}{a} + \frac{b^2}{4a^2} = \frac{-4ac + b^2}{4a^2} +$$ + +Taking the square root of both sides + +$$ +\begin{align*} +x + \frac{b}{2a} &= \frac{\sqrt{-4ac + b^2}}{2a} \\ +x &= \frac{\pm \sqrt{-4ac + b^2} - b}{2a} \\ +&= \frac{-b \pm \sqrt{b^2 - 4ac}}{2a} +\end{align*} +$$ + +This gives you the world famous quadratic formula: + +$$ +x = \frac{-b \pm \sqrt{b^2 - 4ac}}{2a} +$$ diff --git a/Content/posts/2024-03-28-Running-ADFRSuite-on-arm64-Macs.md b/Content/posts/2024-03-28-Running-ADFRSuite-on-arm64-Macs.md new file mode 100644 index 0000000..f43f15c --- /dev/null +++ b/Content/posts/2024-03-28-Running-ADFRSuite-on-arm64-Macs.md @@ -0,0 +1,379 @@ +--- +date: 2024-03-28 20:12 +description: Fixing ADFRsuite on M1/MX chip Macs - CLI Tools +tags: macOS, Cheminformatics +--- + +# Fixing ADFRSuite for Apple Silicon + +We are going to be running everything through Rosetta 2. I am confident that if I had access to the original source code, I could find a way to run everything natively. + +These are the following issues that we will be fixing in this part: + +* “python2.7” cannot be opened because the developer cannot be verified. +* OpenBabel Error +* Segmentation fault while running the [redocking tutorial](https://ccsb.scripps.edu/adcp/tutorial-redocking/) + +For the sake of simplicity, I am assuming that I am running all these commands in the folder `~/Developer/scrippstuff/` + +## Installing Rosetta 2 + +We are going to run all of these steps in the terminal + +```bash +/usr/sbin/softwareupdate --install-rosetta --agree-to-license +``` + +### Install Homebrew for x86 + +Both versions of homebrew (x86 and arm64) can peacefully coexist on your system. + +From now on, every command should be run in a terminal session that starts with this as the first command: + +```bash +arch -x86_64 zsh +``` + +Now, we can install homebrew: + +```bash +/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" +``` + +Here is my output: + +``` bash +➜ scrippstuff uname -a +Darwin Navans-MacBook-Pro.local 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:31:00 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6020 x86_64 +➜ scrippstuff /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" +==> Checking for `sudo` access (which may request your password)... +Password: +==> This script will install: +/usr/local/bin/brew +/usr/local/share/doc/homebrew +/usr/local/share/man/man1/brew.1 +/usr/local/share/zsh/site-functions/_brew +/usr/local/etc/bash_completion.d/brew +/usr/local/Homebrew +==> The following new directories will be created: +/usr/local/Cellar +/usr/local/Caskroom + +Press RETURN/ENTER to continue or any other key to abort: +==> /usr/bin/sudo /bin/mkdir -p /usr/local/Cellar /usr/local/Caskroom +==> /usr/bin/sudo /bin/chmod ug=rwx /usr/local/Cellar /usr/local/Caskroom +==> /usr/bin/sudo /usr/sbin/chown navanchauhan /usr/local/Cellar /usr/local/Caskroom +==> /usr/bin/sudo /usr/bin/chgrp admin /usr/local/Cellar /usr/local/Caskroom +==> /usr/bin/sudo /usr/sbin/chown -R navanchauhan:admin /usr/local/Homebrew +==> /usr/bin/sudo /bin/mkdir -p /Users/navanchauhan/Library/Caches/Homebrew +==> /usr/bin/sudo /bin/chmod g+rwx /Users/navanchauhan/Library/Caches/Homebrew +==> /usr/bin/sudo /usr/sbin/chown -R navanchauhan /Users/navanchauhan/Library/Caches/Homebrew +==> Downloading and installing Homebrew... +remote: Enumerating objects: 47, done. +remote: Counting objects: 100% (47/47), done. +remote: Compressing objects: 100% (19/19), done. +remote: Total 47 (delta 28), reused 47 (delta 28), pack-reused 0 +Unpacking objects: 100% (47/47), 6.11 KiB | 223.00 KiB/s, done. +From https://github.com/Homebrew/brew + + 18ebdd8c8f...67a096fcbb tapioca-compiler-for-tty-rbi -> origin/tapioca-compiler-for-tty-rbi (forced update) +Switched to and reset branch 'stable' +==> Updating Homebrew... +==> Installation successful! + +==> Homebrew has enabled anonymous aggregate formulae and cask analytics. +Read the analytics documentation (and how to opt-out) here: + https://docs.brew.sh/Analytics +No analytics data has been sent yet (nor will any be during this install run). + +==> Homebrew is run entirely by unpaid volunteers. Please consider donating: + https://github.com/Homebrew/brew#donations + +==> Next steps: +- Run these two commands in your terminal to add Homebrew to your PATH: + (echo; echo 'eval "$(/usr/local/bin/brew shellenv)"') >> /Users/navanchauhan/.zprofile + eval "$(/usr/local/bin/brew shellenv)" +- Run brew help to get started +- Further documentation: + https://docs.brew.sh +``` + +At this point, you don't need to edit your `zshrc` or `zsh_profile`. + +### Install pyenv + +The reason we are installing pyenv is because it is easier to build Python 2.7.18 from scratch than messing around with codesigning and quarantine bs on macOS. + +```bash +➜ scrippstuff brew install pyenv +==> Downloading https://ghcr.io/v2/homebrew/core/pyenv/manifests/2.3.36 +############################################################################################################################################################### 100.0% +==> Fetching dependencies for pyenv: m4, autoconf, ca-certificates, openssl@3, pkg-config and readline +==> Downloading https://ghcr.io/v2/homebrew/core/m4/manifests/1.4.19 +############################################################################################################################################################### 100.0% +==> Fetching m4 +==> Downloading https://ghcr.io/v2/homebrew/core/m4/blobs/sha256:8434a67a4383836b2531a6180e068640c5b482ee6781b673d65712e4fc86ca76 +############################################################################################################################################################### 100.0% +==> Downloading https://ghcr.io/v2/homebrew/core/autoconf/manifests/2.72 +############################################################################################################################################################### 100.0% +==> Fetching autoconf +==> Downloading https://ghcr.io/v2/homebrew/core/autoconf/blobs/sha256:12368e33b89d221550ba9e261b0c6ece0b0e89250fb4c95169d09081e0ebb2dd +############################################################################################################################################################### 100.0% +==> Downloading https://ghcr.io/v2/homebrew/core/ca-certificates/manifests/2024-03-11 +############################################################################################################################################################### 100.0% +==> Fetching ca-certificates +==> Downloading https://ghcr.io/v2/homebrew/core/ca-certificates/blobs/sha256:cab828953672906e00a8f25db751977b8dc4115f021f8dfe82b644ade03dacdb +############################################################################################################################################################### 100.0% +==> Downloading https://ghcr.io/v2/homebrew/core/openssl/3/manifests/3.2.1-1 +############################################################################################################################################################### 100.0% +==> Fetching openssl@3 +==> Downloading https://ghcr.io/v2/homebrew/core/openssl/3/blobs/sha256:ef8211c5115fc85f01261037f8fea76cc432b92b4fb23bc87bbf41e9198fcc0f +############################################################################################################################################################### 100.0% +==> Downloading https://ghcr.io/v2/homebrew/core/pkg-config/manifests/0.29.2_3 +############################################################################################################################################################### 100.0% +==> Fetching pkg-config +==> Downloading https://ghcr.io/v2/homebrew/core/pkg-config/blobs/sha256:421571f340277c62c5cc6fd68737bd7c4e085de113452ea49b33bcd46509bb12 +############################################################################################################################################################### 100.0% +==> Downloading https://ghcr.io/v2/homebrew/core/readline/manifests/8.2.10 +############################################################################################################################################################### 100.0% +==> Fetching readline +==> Downloading https://ghcr.io/v2/homebrew/core/readline/blobs/sha256:9796e0ff1cc29ae7e75d8fc1a3e2c5e8ae2aeade8d9d59a16363306bf6c5b8f4 +############################################################################################################################################################### 100.0% +==> Fetching pyenv +==> Downloading https://ghcr.io/v2/homebrew/core/pyenv/blobs/sha256:d117a99ed53502aff29109bfa366693ca623f2326e1e6b4db68fef7b7f63eeba +############################################################################################################################################################### 100.0% +==> Installing dependencies for pyenv: m4, autoconf, ca-certificates, openssl@3, pkg-config and readline +==> Installing pyenv dependency: m4 +==> Downloading https://ghcr.io/v2/homebrew/core/m4/manifests/1.4.19 +Already downloaded: /Users/navanchauhan/Library/Caches/Homebrew/downloads/5b2a7f715487b7377e409e8ca58569040cd89f33859f691210c58d94410fd33b--m4-1.4.19.bottle_manifest.json +==> Pouring m4--1.4.19.sonoma.bottle.tar.gz +🍺 /usr/local/Cellar/m4/1.4.19: 13 files, 739.9KB +==> Installing pyenv dependency: autoconf +==> Downloading https://ghcr.io/v2/homebrew/core/autoconf/manifests/2.72 +Already downloaded: /Users/navanchauhan/Library/Caches/Homebrew/downloads/b73cdb320c4261bbf8d02d03e50dc755c869c5859c1d4e93616898fc7cd939ff--autoconf-2.72.bottle_manifest.json +==> Pouring autoconf--2.72.sonoma.bottle.tar.gz +🍺 /usr/local/Cellar/autoconf/2.72: 71 files, 3.6MB +==> Installing pyenv dependency: ca-certificates +==> Downloading https://ghcr.io/v2/homebrew/core/ca-certificates/manifests/2024-03-11 +Already downloaded: /Users/navanchauhan/Library/Caches/Homebrew/downloads/c431e0186df2ccc2ea942b34a3c26c2cebebec8e07ad6abdae48447a52c5f506--ca-certificates-2024-03-11.bottle_manifest.json +==> Pouring ca-certificates--2024-03-11.all.bottle.tar.gz +==> Regenerating CA certificate bundle from keychain, this may take a while... +🍺 /usr/local/Cellar/ca-certificates/2024-03-11: 3 files, 229.6KB +==> Installing pyenv dependency: openssl@3 +==> Downloading https://ghcr.io/v2/homebrew/core/openssl/3/manifests/3.2.1-1 +Already downloaded: /Users/navanchauhan/Library/Caches/Homebrew/downloads/f7b6e249843882452d784a8cbc4e19231186230b9e485a2a284d5c1952a95ec2--openssl@3-3.2.1-1.bottle_manifest.json +==> Pouring openssl@3--3.2.1.sonoma.bottle.1.tar.gz +🍺 /usr/local/Cellar/openssl@3/3.2.1: 6,874 files, 32.5MB +==> Installing pyenv dependency: pkg-config +==> Downloading https://ghcr.io/v2/homebrew/core/pkg-config/manifests/0.29.2_3 +Already downloaded: /Users/navanchauhan/Library/Caches/Homebrew/downloads/ac691fc7ab8ecffba32a837e7197101d271474a3a84cfddcc30c9fd6763ab3c6--pkg-config-0.29.2_3.bottle_manifest.json +==> Pouring pkg-config--0.29.2_3.sonoma.bottle.tar.gz +🍺 /usr/local/Cellar/pkg-config/0.29.2_3: 11 files, 656.4KB +==> Installing pyenv dependency: readline +==> Downloading https://ghcr.io/v2/homebrew/core/readline/manifests/8.2.10 +Already downloaded: /Users/navanchauhan/Library/Caches/Homebrew/downloads/4ddd52803319828799f1932d4c7fa8d11c667049b20a56341c0c19246a1be93b--readline-8.2.10.bottle_manifest.json +==> Pouring readline--8.2.10.sonoma.bottle.tar.gz +🍺 /usr/local/Cellar/readline/8.2.10: 50 files, 1.7MB +==> Installing pyenv +==> Pouring pyenv--2.3.36.sonoma.bottle.tar.gz +🍺 /usr/local/Cellar/pyenv/2.3.36: 1,158 files, 3.4MB +==> Running `brew cleanup pyenv`... +Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP. +Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`). +``` + +And, build the last version of Python 2.7 + +```bash +➜ scrippstuff PYENV_ROOT="~/Developer/scrippstuff" pyenv install 2.7.18 +python-build: use openssl from homebrew +python-build: use readline from homebrew +Downloading Python-2.7.18.tar.xz... +-> https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tar.xz +Installing Python-2.7.18... +patching file configure +patching file configure.ac +patching file setup.py +patching file 'Mac/Tools/pythonw.c' +patching file setup.py +patching file 'Doc/library/ctypes.rst' +patching file 'Lib/test/test_str.py' +patching file 'Lib/test/test_unicode.py' +patching file 'Modules/_ctypes/_ctypes.c' +patching file 'Modules/_ctypes/callproc.c' +patching file 'Modules/_ctypes/ctypes.h' +patching file 'Modules/_ctypes/callproc.c' +patching file setup.py +patching file 'Mac/Modules/qt/setup.py' +patching file setup.py +python-build: use readline from homebrew +python-build: use zlib from xcode sdk +Installed Python-2.7.18 to /Users/navanchauhan/Developer/scrippstuff/~/Developer/scrippstuff/versions/2.7.18 +``` + +Test the new installation: + +```bash +➜ scrippstuff ~/Developer/scrippstuff/\~/Developer/scrippstuff/versions/2.7.18/bin/python2.7 +Python 2.7.18 (default, Mar 28 2024, 20:47:13) +[GCC Apple LLVM 15.0.0 (clang-1500.1.0.2.5)] on darwin +Type "help", "copyright", "credits" or "license" for more information. +>>> from random import randint +>>> randint(0,10) +6 +>>> exit() +``` + +Now, we can compress this newly created Python version into a `tar.gz` file to replace the one provided in ADFRsuite_x86_64Darwin_1.0.tar.gz. Don't forget the `.` at the end + +```bash +➜ scrippstuff tar -C ./\~/Developer/scrippstuff/versions/2.7.18 -czf new.tar.gz . +``` + +## Install ADFRsuite + +If you don't already have the tarball, you can download it by: + +```bash +$ curl -o adfr.tar.gz https://ccsb.scripps.edu/adfr/download/1033/ +``` + +Uncompress it + +```bash +$ tar -xvzf adfr.tar.gz +``` + +Replace the provided Python archive with the one we created: + +```bash +$ cd ADFRsuite_x86_64Darwin_1.0 +$ mv new.tar.gz Python2.7.tar.gz +``` + +Note: For some reason simply copying it doesn't work and you need to use `mv` + +Just to not mess with anything else, I will be installing everything in a folder called `clean_install` + +```bash +$ mkdir clean_install +$ ./install.sh -d clean_install +... + ADFRsuite installation complete. +To run agfr, agfrgui, adfr, autosite, about, pythonsh scripts located at: +/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/bin +add /Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/bin to the path environment variable in .cshrc or .bashrc: +.cshrc: +set path = (/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/bin $path) + +.bashrc: +export PATH=/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/bin:$PATH +``` + +Now, to test `agfr`, first run the command (replacing `navanchauhan` with yout username) + +```bash +$ export PATH=/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/bin:$PATH +$ agfr +➜ ADFRsuite_x86_64Darwin_1.0 agfr +============================== +*** Open Babel Error in openLib + /Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/lib/openbabel/2.4.1/acesformat.so did not load properly. + Error: dlopen(/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/lib/openbabel/2.4.1/acesformat.so, 0x0009): Library not loaded: /opt/X11/lib/libcairo.2.dylib + Referenced from: <24174F3E-2670-79AC-4F26-F8B49774194A> /Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/lib/openbabel/2.4.1/acesformat.so + Reason: tried: '/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/lib/libcairo.2.dylib' (no such file), '/opt/X11/lib/libcairo.2.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/X11/lib/libcairo.2.dylib' (no such file), '/opt/X11/lib/libcairo.2.dylib' (no such file), '/usr/local/lib/libcairo.2.dylib' (no such file), '/usr/lib/libcairo.2.dylib' (no such file, not in dyld cache) +============================== +``` + + +## Fixing `Open Babel Error` + +```bash +$ brew install cairo +``` + +## Completing the re-docking tutorial + +### Dowbloading the dataset + +```bash +$ curl -o tutorial-data.zip https://ccsb.scripps.edu/adcp/download/1063/ +$ unzip tutorial-data.zip +$ cd ADCP_tutorial_data/3Q47 +``` + +### Conversion to PDBQT + +```bash +$ reduce 3Q47_rec.pdb > 3Q47_recH.pdb +$ reduce 3Q47_pep.pdb > 3Q47_pepH.pdb +``` + +### Preparing Receptor + +``` +$ prepare_receptor -r 3Q47_recH.pdb +$ prepare_ligand -l 3Q47_pepH.pdb +``` + +### Generate Target File + +```bash +$ agfr -r 3Q47_recH.pdbqt -l 3Q47_pepH.pdbqt -asv 1.1 -o 3Q47 +➜ 3Q47 agfr -r 3Q47_recH.pdbqt -l 3Q47_pepH.pdbqt -asv 1.1 -o 3Q47 +Traceback (most recent call last): + File "/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/CCSBpckgs/ADFR/bin/runAGFR.py", line 36, in <module> + from ADFR.utils.runAGFR import runAGFR + File "/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/CCSBpckgs/ADFR/utils/runAGFR.py", line 41, in <module> + from ADFR.utils.maps import flexResStr2flexRes + File "/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/CCSBpckgs/ADFR/utils/maps.py", line 35, in <module> + from ADFRcc.adfr import GridMap + File "/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/CCSBpckgs/ADFRcc/__init__.py", line 34, in <module> + from ADFRcc.adfr import Parameters + File "/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/CCSBpckgs/ADFRcc/adfr.py", line 43, in <module> + import ADFRcc.adfrcc as CPP + File "/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/CCSBpckgs/ADFRcc/adfrcc.py", line 28, in <module> + _adfrcc = swig_import_helper() + File "/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/CCSBpckgs/ADFRcc/adfrcc.py", line 24, in swig_import_helper + _mod = imp.load_module('_adfrcc', fp, pathname, description) +ImportError: dlopen(/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/CCSBpckgs/ADFRcc/_adfrcc.so, 0x0002): Library not loaded: /Users/Shared/mgltoolsDev/src/homebrew/opt/gcc/lib/gcc/8/libgomp.1.dylib + Referenced from: <424BF61E-BF0F-351E-B546-E82EBBD8FBF5> /Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/CCSBpckgs/ADFRcc/_adfrcc.so + Reason: tried: '/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0/clean_install/lib/libgomp.1.dylib' (no such file), '/Users/Shared/mgltoolsDev/src/homebrew/opt/gcc/lib/gcc/8/libgomp.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/Shared/mgltoolsDev/src/homebrew/opt/gcc/lib/gcc/8/libgomp.1.dylib' (no such file), '/Users/Shared/mgltoolsDev/src/homebrew/opt/gcc/lib/gcc/8/libgomp.1.dylib' (no such file), '/usr/local/lib/libgomp.1.dylib' (no such file), '/usr/lib/libgomp.1.dylib' (no such file, not in dyld cache) +➜ 3Q47 +``` + +Sometimes this error is simply outputted as a segmentation fault. But, it is because it cannot find the `libgomp.1.dylib`. I haven't tested using a newer version of GCC to make it work. Building GCC 8 yourself is absolutely painful. We are going to use a copy generated by the homebrew team. + +```bash +$ cd ../../ +$ pwd +/Users/navanchauhan/Developer/scrippstuff/ADFRsuite_x86_64Darwin_1.0 +$ curl -L -H "Authorization: Bearer QQ==" -o gcc8amd64.tar.gz https://ghcr.io/v2/homebrew/core/gcc/8/blobs/sha256:438d5902e5f21a5e8acb5920f1f5684ecfe0c645247d46c8d44c2bbe435966b2 +$ tar -xzf gcc8amd64.tar.gz +$ cp -r gcc@8/8.5.0/lib/gcc/8/* clean_install/lib/ +``` + +Now, we should be able to go back and run the target generation command: + +```bash +$ cd ADCP_tutorial_data/3Q47 +$ agfr -r 3Q47_recH.pdbqt -l 3Q47_pepH.pdbqt -asv 1.1 -o 3Q47 +################################################################# +# If you used AGFR in your work, please cite: # +# # +# P.A. Ravindranath S. Forli, D.S. Goodsell, A.J. Olson and # +# M.F. Sanner # +# AutoDockFR: Advances in Protein-Ligand Docking with # +... +``` + +### Docking Peptide + +```bash +$ adcp -t 3Q47.trg -s npisdvd -N 20 -n 1000000 -o 3Q47_redocking -ref 3Q47_pepH.pdb +``` + +There you have it. Running ADCP on the newest macOS version against all odds. + +I haven't yet looked into fixing/patching `agfrgui` as I don't use the software. But, if someone reallllly needs to run it on Apple Silicon, I am happy to take a look at monkeypatching it. + +In case years down the line the prebuilt version of GCC 8 is not available, let me know so I can replace the link with my mirror. |