I finally got my shit together and migrated my notes from Apple Notes to Obsidian, it turned out to be less painful than I was expecting, BUT it’s a FAF.

For over a decade, I’ve chosen to write all my documents using a Markdown editor of some variety: IAWriter, Bear and now Obsidian because I want access to my files in a format that won’t become obsolete. Markdown is perfect because when all’s said and done, it’s just a text file.

There is one notable exception to this rule, Apple Notes! Notes are so easy to use that I find myself using them by default; I just couldn’t get out of the habit.

There are two main stumbling blocks

1) It’s on all my devices by default
2) It’s where all my notes live

The device problem is pretty simple to overcome; I installed obsidian on all my devices. Stumbling block two has hindered me, though. Getting your notes out of Apple Notes is next to impossible. To do it, I needed to dedicate time, and it just wasn’t high enough on my list of priorities.

That changed this week, though, because I finally connected Obsidian to Readwise to import all my notes highlights from Kindle, Instapaper, Twitter and Apple Books, which I’ve also meant to do for a long time.

This morning I decided to put some time aside and get it done. Here’s a link to the migration guide I used https://forum.obsidian.md/t/import-from-apple-notes-to-obsidian/732. As I already have bear, I decided to go down the following route:

1. Download Bear’s automation workflow
2. Use it to export all notes to HTML pages
3. import the HTML pages into Bear
4. Export the HTML pages as Markdown files
5. Copy the Markdown files into Obsidian.

There are some limitations to this approach as noted in the Bear documentation

- Text, lists, and photos should be included

- **Note** : only macOS 10.15 Catalina supports photo export. On earlier macOS versions, Apple Notes will not export photos

- Task lists convert into bulleted lists - Rich media links will convert to plain text links

- Non-photo attachments like PDFs or other files **are not supported** and will be excluded from export to HTML files. They will remain safely in Apple Notes

All in all, it took around 10 mins to migrate the notes, import, export and import again, but it’s done. Now, hopefully, I can use Obsidian for all my knowledge capture.

31/1/2023

I’ve been super busy these past few weeks, and sadly, my writing has taken a back seat, but I wanted to end the month with a recap of what we achieved.

On 3/1/2023, I wrote in 50: Back to work tomorrow that we wanted to achieve the following things:

• 1. Finish the Retail App
•  2. Complete the migration project
•  3. Complete the Design a better world website
•  4. Completed The Alice Sound Website

So, how did we do?

Successes

1. Finish the Retail App > we pushed the app live on the 24th Jan (68: Retail App), and tomorrow on the 1st Feb it will be comms out to 2500 people in the Retail teams at Three.
2.  Complete the Design a better world website > we are code complete and have handed over the site for Don and Juna to start populating with content to support the publication of his new book.

Partial Success

1. Complete the migration project > this project has been challenging, but today we finalised the migration and reorganisation of the first Sharepoint site. The project is not complete yet, but we have a script that will let us map the old folder locations to new ones and move the files. I’ve been posting the odd snippet about this  (68: Generative AI you need to check its working outs, 67: JavaScript templates strings in Python, 66: Python: MS Graph API Authentication (Without a user)).When I get the chance, I’ll post the script, as it might be helpful to people who need to create and move folders in SharePoint using GraphAPI.
   
2.  Completed The Alice Sound Website > this has been delayed. We’re done with our bit for now, but we look forward to working with the team to finalise the site and release it to the world.

Aside from these goals, the team, led by Dan, have delivered lots of work for Etch, helping them with several clients, and we mustn’t forget the work we’ve been doing with Accenture to support the delivery of the Strata Application.

I’m very proud of the work our team has completed through Jan.

25/01/2023

I’ve mentioned a couple of times that I’ve been playing with ChatGTP to assist with a piece of development work I’m doing.

Today, I validated a new piece of code it presented me with, which looked great and was exactly what I wanted but completely wrong.

I asked it

How do I move a SharePoint folder using Microsoft Graph API in python?

It replied (initially with the correct answer)

import json
import requests

folder_id = "123"
new_parent_id = "456"

url = f"https://graph.microsoft.com/v1.0/drive/items/{folder_id}"
headers = {
    "Authorization": "Bearer your_access_token",
    "Content-Type": "application/json"
}
data = {
    "parentReference": {
        "id": new_parent_id
    }
}

response = requests.patch(url, headers=headers, json=data)

if response.status_code == 200:
    print("Folder moved successfully.")
else:
    print(f"Error moving folder: {response.json()}")

Then I asked Can I move a folder using a path rather than IDs

Where it replied as follows

Yes, it is possible to move a folder in SharePoint using a path rather than IDs using the Microsoft Graph API.

You can use the Move API to move the folder by passing the path of the folder and the new parent folder.

Here is an example of how you can use the requests library to move a folder located at “path/to/folder” to a new parent folder located at “path/to/newparentfolder” :

and supplied the following code

import json
import requests

folder_path = "path/to/folder"
new_parent_path = "path/to/newparentfolder"

url = f"https://graph.microsoft.com/v1.0/drive/root:/{folder_path}:/move"
headers = {
    "Authorization": "Bearer your_access_token",
    "Content-Type": "application/json"
}
data = {
    "destination": {
        "path": new_parent_path
    }
}

response = requests.post(url, headers=headers, json=data)

if response.status_code == 200:
    print("Folder moved successfully.")
else:
    print(f"Error moving folder: {response.json()}")

Unfortunately, this code is not correct! In fact, the official Microsoft Graph API Documentation for the Move method says

Request body

In the request body, supply the new value for the parentReference property. Existing properties that are not included in the request body will maintain their previous values or be recalculated based on changes to other property values. For best performance you shouldn’t include existing values that haven’t changed.

Note: When moving items to the root of a drive, your app cannot use the "id:" "root" syntax. Your app needs to provide the actual ID of the root folder for the parent reference.

Truthfully I’m gutted the code didn’t work because it would have made my life simpler, but it highlights a key consideration when using Generative AI.

You always need to check the output, whether it’s facts, grammar or code. These systems are not full-proof, and it’s incumbent on you to use your critical thinking skills to validate the information you’ve been given. No matter how reasonable it sounds.

24/01/2023

• Good news! After months of toil, we received a security sign-off last week, which means that the Retail App is going live in the next couple of days!
• There is still one feature we’re finishing up, but that doesn’t impact the release.
• The Retail App has, baring a few curve balls, been a good project, and I’m proud of my team for what they’ve built.
• I’m looking to see how Three evolve its use over the coming months, but I’m confident it’s an excellent start for their plans to consolidate the retail employee digital experience.
• Thank you! Callum Ilett, Adam Auckland, Josh Browning, Dave Inoc, Dan Maxwell, Nat Patemkina, Eleanor Morgan and Marcus Woodbridge

23/01/2023

When I first started learning Python, the one thing I wanted more than anything was to be able to use template strings, like had gotten used to in JavaScript.

language = "JavaScript"
string = `this is a string literal in ${language}`

For some reason, I couldn’t find a decent article telling me how to do them.

Then on Sunday, when I asked ChatGTP to generate a piece of code for me, I was struck by something in the output.

url = f"https://login.microsoftonline.com/{TENANT_ID}/oauth2/v2.0/token"

So I asked ChatGTP what the ‘f’ meant, and it told me about ‘f-strings’ for ‘formatted string literals` in python, which is exactly what I was I was looking for.

So here you are. If you want to use template strings in python, then prefix the string with an f, and you’re done!

I’m writing a script to migrate files from one location to another in Sharepoint using MS Graph API.

To use the Graph API, you have to get an authentication token from Azure AD, and there are two ways to do this: 1) Acting on behalf of a user and 2) acting without a user.

Both approaches require you to create an Azure AD App in the Azure portal, but acting on behalf of a user will require an intermediate step of displaying a browser window, so the user can consent to the app acting on their behalf.

For this piece of work, I needed to be able to authenticate without any user involvement. Trying to find good examples of how to do this in Python is devilishly hard, and for a while, I’ve been using option 1, and it’s irked me.

As part of some research into an article I’m writing on Generative AI, I thought I’d see if it could offer a suggestion I could work with.

It did. It gave me the exact answer I was looking for but couldn’t find using GoogleFu!

So here’s the code, in case you’re also looking for it.

import requests

CLIENT_ID = "{your client id}"
TENANT_ID = "{your tenant id}"
CLIENT_SECRET = "{your client secret}" #Don't save this in your code, espically if you're using a public git repo.

url = f"https://login.microsoftonline.com/{TENANT_ID}/oauth2/v2.0/token"
headers = {
    "Content-Type": "application/x-www-form-urlencoded"
}
data = {
    "grant_type": "client_credentials",
    "client_id": CLIENT_ID,
    "client_secret": CLIENT_SECRET,
    "scope": "https://graph.microsoft.com/.default"
}

response = requests.post(url, headers=headers, data=data)

if response.status_code == 200:
    token = response.json()["access_token"]
else:
    print(f"Error getting token: {response.json()}")

The token that’s returned from the /oauth2/v2.0/token will be added as an authorization header to any subsequent call you make to the MS Graph API.

headers = {"AUTHORIZATION": f"Bearer {token}", "Content-type": "application/json"}

What I like about this solution is that it simply uses the request library, making what’s going on more obvious and understandable.

NOTE: I’ve added it to the code, but please don’t keep your client secret in the code. Anyone with access to this can access everything with the scopes allocated to Azure AD App.

2023-01-20

• My good friend Craig turned 55 today! As he’s told me several times, he can now retire.
• He won’t, though, because about 10 years ago, he retrained from being an infrastructure engineer to a FE Lecturer.
• Industries lost, was educations gain!
• He’s a very conscientious lecturer; he thinks deeply about how to give his students the education they need now, not after Uni, to get a job in digital delivery.
• Most of his students go on to Uni, but Craig focuses on helping his students to bridge the gap between theoretical knowledge to practical application.
• He taught a collaborative, agile development process before many companies I know started.
• I’m very grateful to know Craig and to count on him as a friend, and I wish him a very Happy Birthday

2023-01-19

• Polly returned home from the animal hospital today!
• She’s doing well.
• Tam and I have both noticed that she already seems to be doing a lot better walking and going to the toilet, which before the operation was awful.
• She’s still weak and spend most of the evening sleeping.
• We’re hopeful and very grateful to Colin Driver and Dorothy Hajdu at Lumbry park for the wonderful care she received.

Reading: The case for working with your hands by Matthew B Crawford. (100% Complete)

2023-01-19

• A good friend of mine recently wrote an article on focus (in product delivery), aka limiting the work in progress and focusing on one thing to completion.
• Today I had a salient reminder of why it’s so important.
• Let me start by saying that I know how valuable it is to limit your work in progress (WIP) and focus on one thing until completion.
• I purposefully structure my teams to achieve this and manage our backlogs to ensure that the team can follow this maxim.
• However, life rarely runs smoothly when you’re running an agency, and more times than I’d like, we have to manage competing requirements and sometimes, the teams have split objectives.
• We work hard continually to prevent this but sometimes, usually for financial reasons, this isn’t always possible.
• The story I’m telling isn’t in my team directly but with another team, we work closely with.
• In this team, there has been a problem with the deployment pipeline, and the test environment is blocked.
• This has meant the work we’ve been doing has backed up and has impacted some timescales around some quite essential security releases.
• Also, two separate teams have been working on competing outcomes that conflicted with each other.
• When the test environment broke, the other team couldn’t fix it and didn’t reach out for help.
• Rather than stop, reset and figure it out, they doubled down on continuing to make their changes in the hope the next release would fix the issue.
• It didn’t, and the situation worsened.
• We now had more untested code, a broken test environment and a whole bunch of other changes languishing in GitHub branches.
• Eventually, the dam broke, and we got everything unblocked but not in a positive way.
• One impact of all these issues contributed to a member of the other team being signed off work for two weeks with stress. (It wasn’t the only issue, but it contributed to that person being overwhelmed)
• Unfortunately, that person has all the tacit knowledge of how the build and deployment systems work, which meant that the Hail Mary approach to deploying new code to fix the issue couldn’t be deployed!
• It was at this point I was able to step in and reset the work effort.
• We asked a member of our team to investigate the original issue blocking test.
• Three hours later, we found a config file with an error. Once that was fixed, everything else became unblocked.
• The momentum of the other team’s work had narrowed their vision. They decided against the obvious thing and continued with a strategy which compounded the issue.
• There are several significant issues here.
• 1. The configuration issue was in a part of the stack the first team was unfamiliar with. Rather than ask for and accept help, they decided to work around the problems.
• 2. The knowledge needed to complete the critical task was in someone’s head, not the world, for someone else to pick up and resolve.
• 3. The wrong thing was prioritised in the first place. The issue on Test could have only been a config issue, this was apparent weeks earlier when the changes were rolled back to some code known to have worked, and it still didn’t work.
• It’s so easy when you’re in the thick of it to continue regardless. Movement is good, right?
• No, movement isn’t good, momentum is good, and we weren’t building up any.
• It’s better to stop and have some team members under-utilised so the priority issue can be focused on and fixed than to keep going in the hope that it’ll be alright.
• Sometimes it might be alright, but in my experience, you usually end up weeks down the line fixing the thing you should have fixed much earlier.
• We now have two things:
• 1) a working test system to test and deploy the backed-up changes to Prod. Freeing intellectual capacity in the team.
• 2) time to ensure the code written in haste to fix the test environment issue is properly working and tested.

Reading: The case for working with your hands by Matthew B Crawford. (64% Complete)

2023-01-16

• Polly had her surgery today! The vet rung to say that so far everything looked good. The surgery went well, and he’ll call again tomorrow to let us know how Polly is recovering.

• Today went by in a blur. I realised that I had low-level anxiety about Polly’s surgery, which made concentrating a little tricky.
• However, we’re making good progress on our projects.
• The data migration is coming together nicely. The script to reorganise the content is taking longer than expected, but I’m still confident it’ll do what we want it to do.
• The retail app is moving ahead, and we should be able to go live early Feb. We’re still waiting on Info. Sec to sign off, but our plan to resolve the low-impact bugs has been approved in principle, so it’s only a matter of time.
• All in all, things are going well, so there is nothing to grumble about there.
• I think our most significant success is the control we now have over the backlog. It’s now obvious what’s happening and if we have any issues.

Reading: The case for working with your hands by Matthew B Crawford. (64% Complete)

Wordle: 572 5/6