Journal Entry, October 22nd to October 29th 2021

October 29, 2021 · 1 min read

Phoenix Framework

  1. Mon Nov 1 I took the weekend to work through how to add a name column to the scaffolding generated by mix phx.gen.auth to more closely match Laravel Jetstream's profile update flow. In Jetstream, the photo, name, and email address are part of the same change request.

    For gen.auth, email changes are gated behind a validation request happening first. This makes sense, you want to make sure the email is valid and accessible before you change the user to it. The problem with this is that lumping the name into the same changset creates a slight code smell.

    To carve out saving name involves somewhat of a side effect as we modify email_changeset in lib/opportunity_knocks/accounts/user.ex to allow the pattern %{changes: %{name: _}} = changeset -> changeset |> delete_change(:email) in the case statement. This allows name to go through but we push email out of the change in the event both name and email are updated at the same time. This lets us change apply_user_email in lib/opportunity_knocks/accounts.ex to use the snippet:

    |> case do
      %{changes: %{name: _}} = changeset -> changeset |> Repo.update()
      %{changes: %{email: _}} = changeset -> changeset |> Ecto.Changeset.apply_action(:update)

    This is where the smell is particularly strong as I'm doing the same pattern matching to allow the change in the case of name and ignore it for email. I believe as I gain more Elixir knowledge I may refactor this but I also know how deviation can be due to my exposure to Laravel Shift. Shift uses a series of patches to perform framework upgrades. Files that are not altered get changed more easily and when a deviation is detected (merge conflict), Shift goes with its version so you are consistently applying your changes on top of the now improved version. This is the approach we would all take given time or time machines.

Journal Entry, November 27th to December 4th 2020

December 4, 2020 · 1 min read

Laravel

  1. Tue Dec 1

    • Forgot to transfer environment variables for the following:

      QUEUE_CONNECTION=redis
      REDIS_DB=0
      REDIS_CACHE_DB=1
      REDIS_HORIZON_DB=2
    • This caused 2 issues, horizon was no longer split between databases but since 0, 1, and 2 are the defaults this really didn't impact anything.
    • The bigger problem was QUEUE_CONNECTION defaulted to synchronized jobs instead of via redis so Horizon had nothing to do.
    • Getting the environment variables and horizon working didn't show any change, I believe because the data seeded in the redis database was trying to mix earlier version Horizon 2.x with 3.x usage patterns.
    • Flushing the db with flushdb reset horizon and now running jobs via dispatch() show up as expected.

Journal Entry, November 13th to November 20th 2020

November 20, 2020 · 1 min read

MySQL

  1. Tue Nov 17

Journal Entry, November 6th to November 13th 2020

November 13, 2020 · 2 min read

Laravel

  1. Mon Nov 9

    • To introduce a blocklist to filter out apiLog records, we first filter by application_id and then 1 or more arrays of values.
    • This lets us do thing like filter multiple statuses, methods, or uris.
    • Ideally we could take it a step further and filter by only status 422 or DELETE methods but the 1 or more aspect makes things seemingly more difficult.
    • Initially my brain halted using nested conditionals in a "if value is present then check the other fields" repeated.
    • The null coalescing operator is doing this for us by either returning null when nothing is found or the result of the in_array check if present.
    • Because something like null ?? false ?? true returns true, it feels almost hackish but I suspect works for any field we throw at it.
    • My first attempt was to have an array of application_id but this made the Collection->where() usage more difficult.

      {
          "blocklist": [
              {
                  "application_id": ["SAGESYNC", "SAGESYNCMS"],
                  "status": ["422"]
              },
              {
                  "application_id": ["SAGESYNCXC4"]
              }
          ]
      }
    • Ultimately we had to settle on this form as the where() clause looks for values.

      {
          "blocklist": [
              {
                  "application_id": "SAGESYNC",
                  "status": ["422"]
              },
              {
                  "application_id": "SAGESYNCMS",
                  "status": ["422"]
              },
              {
                  "application_id": "SAGESYNCXC4"
              }
          ]
      }
      /**
       * Filter columns against configured blocklist.
       *
       * @param string|null $application_id
       * @param string|null $method
       * @param string|null $uri
       * @param string|null $status
       * @return void
       */
      public function isBlocked($application_id, $method, $uri, $status)
      {
          $result = false;
          $blocklist = collect($this->getBlocklist());
          $blocked = $blocklist->where('application_id', $application_id);
          $blocked->each(function ($block) use (&$result, $method, $uri, $status) {
              $methods = Arr::get($block, 'method', null);
              $uris = Arr::get($block, 'uri', null);
              $statuses = Arr::get($block, 'status', null);
              $filtered = $this->hasValue($methods, $method) ?? $this->hasValue($uris, $uri) ?? $this->hasValue($statuses, $status);
              if (is_null($filtered) || $filtered === true) {
                  // $filtered is true when a value is found or null when all fields are null.
                  $result = true;
              }
          });
      
          return $result;
      }
      
      /**
       * An array has value (has normally checks for keys not values).
       *
       * @param array|null $array
       * @param mixed $value
       * @return boolean
       */
      private function hasValue($array, $value)
      {
          if (is_null($array)) {
              return null;
          }
      
          return in_array($value, $array);
      }
      
      /**
       * Retrieve blocklist section from config file.
       *
       * @return null|array
       */
      private function getBlocklist()
      {
          $localDisk = \Storage::disk('local');
          $contents = $localDisk->get("data/apiLogs/config.json");
          $json = json_decode($contents, true);
      
          if (!is_null($json)) {
              return $json['blocklist'];
          }
      
          return null;
      }

Journal Entry, October 30th to November 6th 2020

November 6, 2020 · 1 min read

Laravel

  1. Mon Nov 2

Elixir

  1. Fri Nov 6

Journal Entry, March 20th to March 27th 2020

March 27, 2020 · 1 min read

MariaDB

  1. Wed Mar 25

    • MariaDB generated JSON columns are possible if we unpack the syntax.

      // $table->string('payload_type')->storedAs('payload->>"$.type"')->after('name');
      $table->string('payload_type')->storedAs("JSON_UNQUOTE(JSON_EXTRACT(`payload`, '$.type'))")->after('name');

Journal Entry, February 7th to February 14th 2020

February 14, 2020 · 1 min read

Laravel

  1. Thu Feb 13

    • Query a group of records based on parameters deep in a parent relation.

      $files = \App\Models\File::whereHas('material', function(\Illuminate\Database\Eloquent\Builder $material) {
          $material->whereHas('category', function(\Illuminate\Database\Eloquent\Builder $category) {
              $category->whereHas('bucket', function(\Illuminate\Database\Eloquent\Builder $bucket) {
                  $bucket->where('path', 'vbs-2020-focus');
              });
          });
      })->toSql();

Journal Entry, January 31st to February 7th 2020

February 7, 2020 · 2 min read

Laravel

  1. Mon Feb 3

    • Mocking multiple overloads based on arguments.

      $this->mock('overload:Stripe\Charge', function ($mock) {
          $mock->shouldReceive('create')
              ->with(\Mockery::on(function ($argument) {
                  return is_array($argument) && isset($argument['source']) && $argument['source'] == 'card_expired';
              }))
              ->times(1)
              ->andThrow(new \Stripe\Error\InvalidRequest("Source should be a valid card id", null)) // This relies on the try/catch logic in the Job
              ->getMock()
              ->shouldReceive('create')
              ->times(1)
              ->andReturn($this->getTestStripeResponse()); // Overload our instance with this default
      });
    • We ignore our raised exception by you guessed it, a try/catch block.

      try {
          $this->withoutExceptionHandling()->artisan("subscriptions:process-remaining --day={$day} --no-progress")
          ->expectsOutput('4 subscription(s) found');
      } catch(\Throwable $exception) {
      }
      
      \Event::assertDispatched(\App\Events\CardCharged::class, 3);
      
      try {
          $this->withoutExceptionHandling()->artisan("subscriptions:process-remaining --day={$day} --no-progress")
          ->expectsOutput('1 subscription(s) found');
      } catch(\Throwable $exception) {
      }
  2. Wed Feb 2

    • Laravel Queue generic exception Illuminate\Queue\MaxAttemptsExceededException
    • This stack trace shows the method call set at 5 maxTries:

      Illuminate\Queue\MaxAttemptsExceededException: App\Jobs\PendingOrderTransferJob has been attempted too many times or run too long. The job may have previously timed out. in /home/forge/obr.rethinkgroup.org/vendor/laravel/framework/src/Illuminate/Queue/Worker.php:394
      Stack trace:
      #0 /home/forge/obr.rethinkgroup.org/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(314): Illuminate\Queue\Worker->markJobAsFailedIfAlreadyExceedsMaxAttempts('redis:horizon', Object(Illuminate\Queue\Jobs\RedisJob), 5)
    • The failed job payload shows 5 attempts were made:

      {"type":"job","timeout":null,"tags":["App\\Model\\PendingOrderStatus:1534"],"id":"3078","data":{"command":"O:32:\"App\\Jobs\\PendingOrderTransferJob\":8:{s:6:\"status\";O:45:\"Illuminate\\Contracts\\Database\\ModelIdentifier\":3:{s:5:\"class\";s:28:\"App\\Model\\PendingOrderStatus\";s:2:\"id\";i:1534;s:10:\"connection\";s:5:\"mysql\";}s:6:\"\u0000*\u0000job\";N;s:10:\"connection\";s:13:\"redis:horizon\";s:5:\"queue\";s:15:\"orders:transfer\";s:15:\"chainConnection\";N;s:10:\"chainQueue\";N;s:5:\"delay\";N;s:7:\"chained\";a:0:{}}","commandName":"App\\Jobs\\PendingOrderTransferJob"},"displayName":"App\\Jobs\\PendingOrderTransferJob","timeoutAt":null,"pushedAt":"1580851619.0033","maxTries":null,"job":"Illuminate\\Queue\\CallQueuedHandler@call","attempts":5}

Journal Entry, December 13th to December 20th 2019

December 20, 2019 · 1 min read

Laravel

  1. Mon Dec 16

    • Reminder of JSON fields

      $table->json('source_payload');
      $table->string('order_id')->storedAs('source_payload->>"$.eCommerceOrderId"');
      $table->string('user_id')->storedAs('source_payload->>"$.eCommerceCustomerId"');
      $table->string('organization_id')->storedAs('source_payload->>"$.obrorgid"');
      
      // See https://themsaid.com/laravel-mysql-json-colum-fast-lookup-20160709/
      $table->index('order_id');
      $table->index('user_id');
      $table->index('organization_id');
      SELECT
          *, payload->>"$[0].eCommerceCustomerId" as `eCommerceCustomerId`
      FROM
          pendingOrderStatus
      WHERE
          payload->>"$[0].eCommerceCustomerId" = '16350';

Journal Entry, November 15th to November 22nd 2019

November 22, 2019 · 1 min read

Laravel Nova

  1. Mon Nov 18

    • The following is necessary as name isn't a publicly accessible property facepalm.
    • Using name is allowed oddly enough but sorting isn't respected so my guess is Laravel can't find the property and tries to sort by the object instead.
    • It's extremely off-putting to see it partially sort and have no fucking clue why it didn't fully sort as expected. I spent longer than I should've and hated myself when I found the answer.

      return $categories->sortBy(function(\App\Classes\Uploads\Category $category) {
          return $category->getName();
      }, SORT_NATURAL);