Posts

Running Keycloak with Postgres in Docker

April 2, 2023 · 3 min read

I was recently tasked with evaluating Laravel-based and external SSO workflows and stumbled on Keycloak in several places. There is a Socialite provider as well as Supabase auth so I needed a way to make a quick evaluation. I chose to use Docker as I didn't want to dive into the world of Java installations. There is some extensive documentation, but like some OSS projects, it can be a firehose at times.

Fortunately, Google came to the rescue with many resources I've included at the bottom of this post for reference. A Docker compose file to spin up the Keycloak container and Postgres to store its data. The docker image needs environment variables set and the best way I know to do that is through direnv and specifically the asdf version manager plugin for it.

docker-compose.yml

I lifted this template from this StackOverflow post and surgically altered it for my purposes. I commented out the parts that weren't relevant and stripped away the backend and frontend services since I no longer needed them.

---
version: "3.8"

services:
  database:
    image: postgres:14
    container_name: database
    environment:
    # add multiple schemas
      # POSTGRES_MULTIPLE_DATABASES: ${DB_DATABASE},${KEYCLOAK_DATABASE}
      POSTGRES_DB: ${DB_DATABASE}
      POSTGRES_USER: ${DB_USER}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      # POSTGRES_KEYCLOAK_USER: ${KEYCLOAK_USER}
      # POSTGRES_KEYCLOAK_PASSWORD: ${KEYCLOAK_PASSWORD}
      # POSTGRES_DB2: ${KEYCLOAK_DATABASE}
    hostname: local
    restart: always
    volumes:
      - ./db-data:/var/lib/postgresql/data/
      - ./sql:/docker-entrypoint-initdb.d/:ro
      # - ./sql/access_attempt.sql:/docker-entrypoint-initdb.d/A.sql
      # - ./sql/bceid.sql:/docker-entrypoint-initdb.d/B.sql
      # - ./sql/lookup_activitytype.sql:/docker-entrypoint-initdb.d/C.sql
      # - ./sql/lookup_gender_pronoun.sql:/docker-entrypoint-initdb.d/D.sql
      # - ./sql/client.sql:/docker-entrypoint-initdb.d/E.sql
    ports: 
      - "5439:5432"
    networks:
      - db-keycloak

  keycloak:
    image: quay.io/keycloak/keycloak:21.0.1
    command: ["start-dev"]
    container_name: keycloak
    environment:
      DB_VENDOR: ${DB_VENDOR}
      DB_ADDR: database
      DB_PORT: 5432
      DB_SCHEMA: public
      DB_DATABASE: ${DB_DATABASE}
      DB_USER: ${DB_USER}
      DB_PASSWORD: ${DB_PASSWORD}
      KEYCLOAK_USER: ${KEYCLOAK_USER}
      KEYCLOAK_PASSWORD: ${KEYCLOAK_PASSWORD}
      KEYCLOAK_ADMIN: ${KEYCLOAK_ADMIN}
      KEYCLOAK_ADMIN_PASSWORD: ${KEYCLOAK_ADMIN_PASSWORD}
      KC_PROXY_MODE: edge
      KC_METRICS_ENABLED: true
      KC_HTTP_ENABLED: true
    ports:
      - "8089:8080"
      - "8443:8443"
    depends_on:
      - database
    restart: always
    links: 
      - database
    networks:
      - db-keycloak

networks:
  db-keycloak:
    driver: bridge

.env

This sets the environment variables used by both Postgres and Keycloak.

APP_DOMAIN="localhost"
DB_VENDOR="postgres"
DB_DATABASE="keycloak"
DB_USER="keycloak"
DB_PASSWORD="keycloak"
KEYCLOAK_USER="developer"
KEYCLOAK_PASSWORD="developer"
KEYCLOAK_ADMIN="admin"
KEYCLOAK_ADMIN_PASSWORD="admin"
KC_DB="postgres"
KC_DB_URL="jdbc:postgresql://database/keycloak"
# KC_HOSTNAME_FRONTEND_URL=""
# KC_HOSTNAME_ADMIN_URL=""

Usage Instructions

  1. Run docker compose up to run the containers in interactive mode.

    1. Upon first run your db-data directory should fill up with files and directories.
  2. Run docker compose up -d to run your containers in the background.
  3. Run docker-compose down --rmi all to completely cleanup all containers.
  4. Modify the command command: ["start-dev"] to start keycloak in the other modes. This is necessary as the entrypoint isn't specific enough.

It may be useful to create an optimized Keycloak image, but that wasn't necessary for my purposes.

  1. Modify the image section image: quay.io/keycloak/keycloak:21.0.1 to keycloak-custom:latest to use the custom image.

    1. We can publish this image to our own registry if we wanted but this is fine for now. I doubt we'll want to use this outside of production.
  2. To build the optimized Keycloak image independently, run the following: docker build -t keycloak-custom:latest ..

Resources

Attending the DockYard Academy Beta Cohort

March 20, 2023 · 4 min read

I had the privilege of hanging around Brooklin Myers before he joined DockYard as an instructor in early 2022. A unique Elixir community slowly coalesced with the first video of the beta cohort starting on September 21st, 2022. I wouldn't join the group until October 15th as I wasn't sure what to make of it at first. I figured I would audit the class like I was some college kid.

The Curriculum

The academy skews toward junior developers or other Elixir newbies without previous formal instruction. Despite that, the curriculum and the commitment of 2 hours per day was an exceptional resource regardless of experience level.

The curriculum is not as lightweight as Elixir koans, and it is not as self-paced as Exercism's Elixir track. I hadn't been a part of the Exercism Elixir cohort on Discord, but I suspect it may have been similar.

What sets the curriculum apart is that it starts in Livebook, a low barrier to entry for learning Elixir. Eventually, it moves to bare mix new projects, graduating to full-on mix phx.new Phoenix applications. The beta curriculum experience was different than the first cohort, and there are upcoming changes for the second cohort. It's helpful to know the curriculum changes when pain points surface. There is no sleight of hand or abandonware as the official repository is what is taught from start to finish.

As someone that can have analysis paralysis at times when it comes to what and how to learn, having the path chosen for me was extremely helpful. Exercism gates the syllabus, but that can be daunting to decipher when you're starting. I also rushed through the concepts I was interested in rather than taking the time to enjoy the journey. I firmly believe the curriculum and Exercism complement each other very well.

The curriculum culminated in a capstone project, a chance to bundle all the skills we learned to produce our applications. The capstone sets it apart from other learning materials.

The Cohort

The beta cohort was a mix of Elixir newbies, seasoned Elixir developers and mentors, and people that hadn't touched a programming language. We experimented with teaching styles and nailed a cadence "locked in" at the last minute. Everyone I paired with showed remarkable improvement between October and the demo day on January 20th. That level of improvement is a testament to Brooklin's teaching style. Fundamentals became second nature very quickly. I would be lucky to work with anybody I met in the cohort or Discord server, as everyone grew into a developer. Elixir has a way of binding cohesive communities, but Brooklin truly has his superpower with the people around him. As much as I love DockYard, this felt like "The Brooklin Show" *sponsored by DockYard(tm)

Demo Day

I was one of the fewer resident developers to present on Demo Day, and that almost didn't happen. My capstone project, Beatseek, was hastily thrown together by duct tape. I had a working prototype at least a month before the deadline, but I had only given myself ten days from mix phx.new to what I presented. I thought it went well without a script, working through some prior presentations, but it was unpolished. I used sleight of hand as I do on some demos, but as a magician, I wanted to show all the tricks.

I didn't cut a public release until two months after demo day because I wasn't happy with what I produced. I had to retrofit tests, which exposed several shortcomings. If I had to do it again, I would choose anything other than id3 tags because the edge cases are absurdly complex.

What Would I Change?

I had a few issues working through the curriculum or with other cohort members. Tracking progress was difficult, but I used an Obsidian daily standup journal template to check off the table of contents manually. The standup journal became a good way of tracking changes over time, though there were few. The ramp-up to Phoenix for people with no web development or API exposure was pretty steep for the beta cohort, but I don't know if this is still true. Web development fundamentals span a breadth of knowledge, but the curriculum helps cement these concepts. People new to web development may wish to spend more time going through the same sections a few times until the concepts of things like MVC are less foreign. It'll make the later parts much easier to push through.

The End Result

I am 100% glad I had access to an instructor and mentor, even in a limited capacity. Everyone on the Discord server is excellent and a joy to be around. I would do this again in a heartbeat, but 2 hours was a sweet spot for someone like me with a full-time position to juggle. I can see how much more beneficial the 6-hour full day could be with more immersion, but that is a lot of material to cram. We had some luxury in drawing the material out and taking some time to keep everyone on the same pace.

Do you want to build an ElixirLS?

January 11, 2023 · 3 min read

There are certain instances where you may want to build ElixirLS to run against the version of Elixir/Erlang that you're using. The recommendation from the package is:

If you're packaging these archives in an IDE plugin, make sure to build using the minimum supported OTP version for the best backward-compatibility If you're like me, you may not care to support older versions of Elixir. How do we configure the plugin to run the latest version?

The output I see in VSCode's Output tab (Shift-Command-U on macOS) for the ElixirLS extension:

[Info  - 4:33:53 PM] Started ElixirLS v0.13.0
[Info  - 4:33:53 PM] ElixirLS built with elixir "1.12.3" on OTP "22"
[Info  - 4:33:53 PM] Running on elixir "1.14.2 (compiled with Erlang/OTP 25)" on OTP "25"
[Info  - 4:33:53 PM] Elixir sources not found (checking in /home/build/elixir). Code navigation to Elixir modules disabled.
[Info  - 4:33:54 PM] Loaded DETS databases in 32ms
[Info  - 4:33:54 PM] Starting build with MIX_ENV: test MIX_TARGET: host
[Info  - 4:33:55 PM] Compile took 854 milliseconds

There are numerous articles on building from source. What if we'd prefer to build the extension instead?

Let's unpack that Docker command to perform each step:

  1. Clone the repository with the latest branch: git clone --recursive --branch v0.13.0 https://github.com/elixir-lsp/vscode-elixir-ls.git /tmp/vscode-elixir-ls.
  2. Change to our temp directory: cd /tmp/vscode-elixir-ls.
  3. Install npm dependencies: npm install.
  4. Change to the elixir-ls directory: cd elixir-ls.
  5. Install Elixir dependencies: mix deps.get.
  6. Change to the parent directory: cd ...
  7. Package the extension: npx vsce package.
  8. Make an extensions directory in $HOME: mkdir -p $HOME/extensions.
  9. Copy the extension: cp /tmp/vscode-elixir-ls/elixir-ls-0.13.0.vsix $HOME/extensions.
  10. Remove the temporary directory: rm -rf /tmp/vscode-elixir-ls.

It is crucial to install Elixir v1.14.x and Erlang 25.1.x using your favorite method prior to packaging the new extension. I'm using asdf global to do this, but you could create a local .tool-versions inside the tmp folder if you wish. The extension should now live at /tmp/vscode-elixir-ls/elixir-ls-0.13.0.vsix.

The remaining steps copy the package to a directory the Docker container knows, and it's okay to stop here. Because the prepublish.bash file that executes at step #7 runs mix deps.get, we can eliminate steps 4, 5, and 6. These commands also compile the extension using MIX_ENV=dev, which we may not want. To change this, we can edit the last line in prepublish.bash to MIX_ENV=prod mix elixir_ls.release -o ../elixir-ls-release to compile for production.

Putting all of the (now reduced) commands together:

git clone --recursive --branch v0.13.0 https://github.com/elixir-lsp/vscode-elixir-ls.git /tmp/vscode-elixir-ls
cd /tmp/vscode-elixir-ls
npm install
npx vsce package
mkdir -p $HOME/extensions
cp /tmp/vscode-elixir-ls/elixir-ls-0.13.0.vsix $HOME/extensions
rm -rf /tmp/vscode-elixir-ls

We can install the extension from the VSIX file using the UI or the command code --install-extension $HOME/extensions/elixir-ls-0.13.0.vsix. To take advantage of the new extension in our projects, we need to rm -rf .elixir_ls and navigate to an Elixir file. ElixirLS won't start compiling until an Elixir file is open in the editor, and it'll usually take a few minutes to rebuild everything.

With the new extension installed we should see the change in VSCode's Output tab:

[Info  - 4:35:42 PM] Started ElixirLS v0.13.0
[Info  - 4:35:43 PM] ElixirLS built with elixir "1.14.2" on OTP "25"
[Info  - 4:35:43 PM] Running on elixir "1.14.2 (compiled with Erlang/OTP 25)" on OTP "25"
[Info  - 4:35:43 PM] Elixir sources not found (checking in /home/build/elixir). Code navigation to Elixir modules disabled.
[Info  - 4:35:48 PM] Loaded DETS databases in 414ms
[Info  - 4:35:48 PM] Starting build with MIX_ENV: test MIX_TARGET: host
[Info  - 4:35:49 PM] Compile took 1811 milliseconds

asdf Seems Broken After Homebrew Upgrade

January 9, 2023 · 1 min read

I upgraded the excellent asdf version manager using Homebrew and ran into a snag when trying to perform mix commands. I encounterd the error /Users/jbrayton/.asdf/shims/mix: line 13: /usr/local/Cellar/asdf/0.10.2/libexec/bin/asdf: No such file or directory. The key to notice here is the path /usr/local/Cellar/asdf/0.10.2/ when the newest version is 0.11.0, as there is clearly a mismatch. I restarted my terminal and shell, but the problem persisted. I noticed all the files in ~/.asdf/shims had the line exec /usr/local/Cellar/asdf/0.10.2/libexec/bin/asdf exec "odbcserver" "$@" # asdf_allow: ' asdf '. This line is not what we wanted and indicates the problem.

After looking at the pinned https://github.com/asdf-vm/asdf/issues/785 and then following that to https://github.com/asdf-vm/asdf/issues/1393, the solution rm -rf ~/.asdf/shims; asdf reshim fixes my problem. Now, whenever I examine one of the shim files, I see the line exec /usr/local/opt/asdf/libexec/bin/asdf exec "mix" "$@" # asdf_allow: ' asdf ' as expected. The directory /usr/local/opt is what I see when I run the command brew --prefix asdf as the prefix is no longer /usr/local/Cellar/asdf/0.10.2/ or the Cellar location. This corrective measure should be a more permanent solution moving forward as the prefix /usr/local/opt should no longer change in the future.

This issue was also somewhat of a perfect storm as Phoenix 1.7 rc.1 dropped two days ago and I had just upgraded a bunch of homebrew packages, including asdf.

Laravel Valet for Production Domains

January 8, 2023 · 2 min read

Recently, after a brief outage at work, I wondered if it would be possible to replicate the problem locally using Laravel Valet. My Google search landed on this StackOverflow post, where the answers shot down the idea. Not to be dissuaded by something I read on the internet, I started investigating if it was possible and stumbled upon what I think is a viable solution. There aren't very many hoops to jump through or major quirks so I believe it's not only possible but could be supported out of the box.

In my case, I want to proxy the domain scdn-app.thinkorange.com through my local version of the Laravel application.

  1. Edit ~/.config/valet/config.json on macOS and change the tld parameter from test to com.
  2. Change to the directory of your application.
  3. Run the command valet link scdn-app.thinkorange to set up our valet configuration to point the domain to this directory.
  4. Run the command valet secure scdn-app.thinkorange to set up the SSL certificate.
  5. Change the directory to dnsmasq cd ~/.config/valet/dnsmasq.d.
  6. Copy the existing TLD config to cover the .com domain with the command cp tld-test.conf tld-com.conf.
  7. Edit the new file to change the first address line to address=/.com/127.0.0.1 and save the file.
  8. (Optionally) Isolate the site to PHP 8.1 with the command valet isolate --site scdn-app.thinkorange php@8.1.
  9. Change your /etc/hosts file to redirect the domain to 127.0.0.1 for ipv4 and ::1. I use the excellent Gas Mask to make this step easier.

Now we should have a functional production proxy through our local machine. This configuration creates a few problems around keeping the com TLD. Fortunately, a few extra steps are necessary for us to switch back to .test while also keeping this site functional.

  1. Edit ~/.config/valet/config.json again and change the tld parameter from com back to test. This change will immediately break our site.
  2. Change to the Sites directory cd ~/.config/valet/Sites.
  3. If we use ls -al to list the directory, we'll see our site scdn-app.thinkorange. Let's change that.
  4. Run the command mv scdn-app.thinkorange scdn-app.thinkorange.com.

Our site should now be working again. We are also able to continue serving our previous local test domains.

Because we can create a permanently functional system using these steps, I believe it should be possible to create a pull request to reduce the number of hoops we have to jump through. I'd love to be able to run valet link scdn-app.thinkorange.com. with a period at the end to denote I'm including the full domain with TLD. That would eliminate the temporary step of editing the config.json file, and the Sites directory would just work(TM) as it would include the .com directory name. I don't believe we even need the dnsmasq changes as I'm able to navigate to a functional site without them. I believe Gas Mask is doing the work, but it's better to be safe than sorry.

If you'd prefer a YouTube video where I stumble through recreating these steps from scratch: