interface User {    name: string    age: number    address: {        city: string        country: string    }}

TS enables us to map over the keys of an object type to derive new types. Simplest example would be something like this:

type User$1 = {    [K in keyof User]: User[K]}

Because we are mapping keys in User type to corresponding values in User type, our User$1 is equivalent to User.

Most mapped types we see in the wild will use a different value type, typically one that is derived from User[K].

The issue we want to tackle is that we want to omit certain keys when generating the mapped type.

Here, let's say we want to include only those keys whose value types are primitive.

type Primitive = string | number | boolean | symbol;

If the user type is known to us in advance, we can simply do:

type User$2 = {    [K in keyof Pick<User, "name" | "age">]: User[K]}// { name: string, age: number }

However, often we won't have the User type known to us ahead of time, or we wouldn't want to hard code the keys.

We could map the values we don't want to never:

type User$3 = {    [K in keyof User]: User[K] extends Primitive        ? User[K]        : never}// { name: string, age: number, address: never }

This is close. But, that never type can cause issues in some scenarios. By mapping the value type to never, we are saying that forany type to be compatible with User$3, address must not be present. Depending on the use case, this can be undesirable.

What if we want to just remove the keys which we don't want ?

To achieve that, instead of mapping the value type, we can map the key type to never for cases we want to omit:

type User$4 = {    [K in keyof User as (        User[K] extends Primitive             ? K             : never    )]: User[K]}// { name: string, age: number }

This is a bit less readable, but exactly what we wanted.

However, sooner or later, the primary taskfile grows quite large as we add more and more tasks.

Fortunately taskfile has a nice solution for this, we can include other taskfiles in a taskfile.

version: "3"includes:  backend:    taskfile: ./modules/backend/Taskfile.part.yaml    dir: ./modules/backend  frontend:    taskfile: ./modules/frontend/Taskfile.part.yaml    dir: ./modules/frontend

This is particularly convenient for monorepos where each module will typically have a set of closely related set of tasks that depend on each other, but are otherwise unrelated to tasks for other modules/services. Its also nice that all the tasks get auto-prefixed so a build task defined in backend/Taskfile.part.yaml will be available as backend:build.

Note that we named our constituent taskfiles as Taskfile.part.yaml rather than Taskfile.yaml.

If we use the latter, any tasks defined in the top level Taskfile.yaml will become inaccessible while we are in child directories because task will just find the first Taskfile.yaml in parent directory and stop there. So any tasks in parent taskfile will not be found. It also causes confusion because in subdirectories we will need to use unprefixed tasks where as in top level we will need to run prefixed tasks.

If you use yaml-language-server, you can get autocompletion and checking for your Taskfile.part.yaml files by adding this comment to top:

# yaml-language-server: $schema=https://taskfile.dev/schema.json

However, recently metro has added support for symlinks (though marked unstable as of this writing) which makes it possible for us to use a react-native + metro project with pnpm.

There are a few considerations though:

1. You may need to install some additional dev dependencies which didn't before.

   There are some dependencies which the RN project generated by the scaffolder uses, but doesn't depend on directly. They are just assumed to be available due to hoisting behavior of npm.

   pnpm adopts a more principled stance towards dependency hosting, which IMHO is better. So, you will need directly depend on these packages.

   pnpm i -D @react-native/gradle-plugin @react-native-community/cli-platform-android @react-native-community/cli

   Be mindful of versions if you are updating an older project that is not yet updated to use latest RN.

2. Metro still needs to be aware of all source roots. This is an issue if we are also using pnpm workspaces.

   While the bundler can follow symlinks, we still need to explicitly register all the root directories which may contain bundled files (including the symlink targets) through the rather weirdly named watchFolders.

   From the official docs

   Despite the naming of this option, it isn't related solely to file watching. Even in an offline build (for example, in CI), all files must be visible to Metro through the combination of watchFoldersand projectRoot.

   So let's say your RN project depends on another module in the pnpm workspace called shared, we need the following in metro.config.js:

   module.exports = mergeConfig(defaultConfig, {    // ... other config    watchFolders: [        __dirname,        path.resolve(__dirname, "../../node_modules"), // Should resolve to top level node_modules        path.resolve(__dirname, "../shared"), // Any other modules the RN project depends on    ],}

   As shown above, we also need to add the top level node_modules folder where pnpm will install the dependencies.

And that is pretty much all that is needed.

This post outlines the quick setup that I have found to be effective.

Rainbow CSV plugin

Rainbow CSV is a nice plugin (available for many editors) which colors the csv in a different color for each column. This makes it easy to distinguish between values in different columns.

Here is what this looks like editing an example file:

Another convenient feature is :RainbowAlign which vertically aligns the columns.

Be careful that this is actually adding spaces in the file contents, so anything that consumes your csv will likely need to trim the cell contents.

Its also super convenient that as we move about in the csv, the footer shows the column we are currently at:

The plugin also supports an inbuilt query language but for anything more complex than basic search, I find myself reaching to SQLite which makes it really easy to import csv files into a table.

Frozen headers

If you want a slightly more spreadsheet like feel with the header frozen on top, we can achieve that using core vim features.

We can use :sp to horizontally split the screen into two windows, and then use Ctrl+w 1 _ to resize the top one to show single row.

After that we can :set scrollopt=hor to configure scroll binding behavior to horizontal sync and after that :set scrollbind in both windows will ensure that as we move about in the primary window the headers stay in sync.

However, we sometimes forget that in javascript arrays are objects too, and thus object destructuring can be used with arrays as well.

const { 0: a } = [10]console.log(a) // 10

We need to provide an alternative identifier because 0 can not be used as one. This works because js arrays quack like objects with string keys.

console.log([10]["0"]) // 10

Above usages are not so useful though, but it can be handy in scenarios where you get a collection from somewhere else and want to extract something useful out of it succinctly.

So instead of:

[1, 2, 3].map((item, index, collection) => `${index}/${collection.length}`) // ['0/3', '1/3', '2/3']

We could write:

[1, 2, 3].map((item, index, { length }) => `${index}/${length}`) // ['0/3', '1/3', '2/3']

Or, let's say we need the siblings of each item while iterating:

[1, 2, 3].map((item, index, {     [index-1]: prev,    [index+1]: next,}) => ({ item, prev, next }))// [{"item":1,"prev": undefined, "next":2},{"item":2,"prev":1,"next":3},{"item":3,"prev":2, "next": undefined}]

Here we take advantage of the feature that while destructuring, our keys can be computed expressions.

Also, there is nothing stopping us from making use of destructured items in key expressions within the same destructuring assignment:

[1, 2, 3].map((current, index, {     length,    0: first,    [length-1]: last}) => `${first}...${current}...${last}`)// ['1...1...3', '1...2...3', '1...3...3']

JS is fun, yeah ?

This quick recipe illustrates how to integrate hono with graphql-yoga to serve a GraphQL API. Yoga is quite flexible about the underlying request handling library.

import { serve } from "@hono/node-server"import { createYoga } from "graphql-yoga"import { Hono } from "hono"import { getConfigParam } from "./config.js"import { schema } from "./schema.js"const app = new Hono()const yoga = createYoga({  schema,  logging: "debug",  plugins: [/* Add any plugins here */],})// Configure hono to delegate graphql requests to GraphQL Yogaapp.use("/graphql", async (context) =>  yoga.handle({    request: context.req.raw,  }, {}))serve(app, (info) => {  console.log(`Listening on http://localhost:${info.port}`) // Listening on http://localhost:3000})

schema being imported here is a GraphQLSchema instance. You can generate it through any graphql-js compatible library that you are familiar with.

If you don't have a preference already, I recommend Garph. It makes it quite convenient to define type safe GraphQL schemas in pure typescript without needing any code generation. And of course, it plays well with graphql-yoga.

import { g, InferResolvers, buildSchema } from 'garph'import { createYoga, YogaInitialContext } from 'graphql-yoga'import { createServer } from 'http'const queryType = g.type('Query', {  greet: g.string()    .args({      name: g.string().optional().default('Max')    })    .description('Greets a person')})const resolvers: InferResolvers<{ Query: typeof queryType }, { context: YogaInitialContext }> = {  Query: {    greet: (parent, args, context, info) => `Hello, ${args.name}`  }}const schema = buildSchema({ g, resolvers })

However, sometimes when these type compositions get too complex, we can get run into this type error:

error TS7056: The inferred type of this node exceeds the maximum length the compiler will serialize. An explicit type annotation is needed.

This problem happens because when we are composing these schema definitions (zod here), for example:

const Department = z.object({    name: z.string()})export const Company = z.object({    name: z.string(),    departments: Department.array(),});

While we are using a composition of two schema definitions, when we infer the type we will get a single complex type.

type IDepartment = z.output<typeof Department>;// { name: string }type ICompany = z.output<typeof Company>// { name: string, departments: { name: string }[] }

Note that ICompany type does not make use of IDepartment (this would happen even if we defined these types as interfaces instead of aliases). The inferred type of each schema definiton is a separate self contained type.

So when we are composing many complex types, the composition's serialized representation includes information about every field of every composed type. This is fine for the simple case here, but it can grow large rapidly for complex types.

Typescript compiler has an upper limit on the size of this serialized representation. When this is exceeded we get the above error.

The solution to this is quite simple. We can reduce the size of the composed type by introducing interfaces for the types it uses.

const Department = z.object({    name: z.string()})type IDepartment = z.output<typeof Department>;type IDepartmentIn = z.input<typeof Department>;// Explicit type for the runtime schema of Departmentexport interface IDepartmentRT extends z.ZodType<IDepartment, z.ZodTypeDef, IDepartmentIn> {}export const Company = z.object({    name: z.string(),    departments: (Department as IDepartmentRT).array(),});type ICompany = z.output<typeof Company>;type ICompanyIn = z.input<typeof Company>;

Now the serialized representation of ICompany will refer to IDepartmentRT and so the fields of Department type are not included in the serialized representation, reducing its size. And we also don't have to redefine the zod schema separately as an interface.

This post outlines an alternative solution. If we structure our input parameter as json (or any other compound data type for that matter), we can handle the conditional clauses within SQL with some creativity.

For example, to find users by id or email, we can do something like this:

select * from "user" u left join user_email ue     on ue.user_id = u.idwhere     (        -- Filter conditions        ($1::jsonb #>> '{id,eq}' is null or u.id = ($1::jsonb #>> '{id,eq}')::bigint ) or        ($1::jsonb #>> '{email,eq}' is null or ue.email = ($1::jsonb #>> '{email,eq}'))    ) and    -- Fallback to ensure that nothing is selected if no filters are passed    ($1::jsonb #>> '{id,eq}' is not null or $1::jsonb #>> '{email,eq}' is not null)

We can pass input parameters like: { "id": { "eq": 1 } } or { "email": { "eq": "lorefnon@tutanota.com" } } and it will filter appropriately.

We can skip the jsonb cast if our client library allows us to cast parameters while passing to database

select * from "user" u left join user_email ue     on ue.user_id = u.idwhere     (        -- Filter conditions        (($1 #>> '{id,eq}') is null or u.id = ($1 #>> '{id,eq}')::bigint ) or        (($1 #>> '{email,eq}') is null or ue.email = ($1 #>> '{email,eq}'))    ) and    -- Fallback to ensure that nothing is selected if no filters are passed    ($1 #>> '{id,eq}' is not null or $1 #>> '{email,eq}' is not null)

While this is arguably more verbose and less readable (due to the use of non-intuitive json operators), there are a few benefits to this approach.

One is that we don't need a complex query builder library, which may be a plus if we are working with a niche language and/or a restricted runtime like openresty.

Other is that this approach pairs well with prepared statements and the query can be parsed and analyzed just once - if we use a query builder to create slightly differing SELECT statements for various use cases, they would need to be parsed separately each time. Of course, we are trading off per-execution query parsing with per-execution json parsing, but if our json is not very complex, the parsing overhead is lower.

One apparent limitation is that we can not perform conditional joins. However, that can often be alleviated with usage of unions:

select * from "user" uwhere     ($1 #>> '{id,eq}') is not null and u.id = ($1 #>> '{id,eq}')::bigintunion all select u.* from "user" ujoin user_email ue    on ue.user_id = u.idwhere    ($1 #>> '{email,eq}') is not null and ue.email = ($1 #>> '{email, eq}')

or, by building up subselections for individual filters in CTE steps and then combining them in the final select:

-- CTE Steps for each possible criteriawith user_fby_email as (    select ue.user_id     from user_email ue    where ($1::json #>> '{email,eq}') is not null and ue.email = ($1::json #>> '{email,eq}'))-- Directly handle the criteria for user table (for which an additional subquery is unnecessary)select u.*from "user" uwhere (($1 #>> '{id,eq}' is not null and u.id = ($1::jsonb #>> '{id,eq}')::bigint ))union all-- Merge users for each criteriaselect u.*from "user" uwhere u.id in (select user_id from user_fby_email)

The latter tends to be more readable when there are many possible filter criteria.

When using multiple tables, these approaches are likely to enable better index utilization.

We do need to analyze EXPLAIN queries with all possible combinations of inputs to ensure that indexes are properly utilized - but that is something we would have needed when using query builders or ORMs too.

This post is a quick recipe to funnel docker compose logs to AWS S3 for long term archival.

The utility we use for shipping the logs is Vector. Vector is a full featured observability pipeline solution which supports not just shipping but also aggregating and transforming logs. However, in this post we just use it to ship logs to a bucket so that we can analyse them later if needed. Being a rust-based native utility, it has a very low footprint and is well suited to single server or homelab deployments.

In docker-compose.yaml:

version: "3.9"services:  vector:    image: timberio/vector:0.31.0-debian    restart: always    container_name: vector    hostname: vector    environment:       - DOCKER_HOST="unix:///var/run/docker.sock"      - AWS_ACCESS_KEY_ID      - AWS_SECRET_ACCESS_KEY      - AWS_REGION    ports:      - '8383:8383'    volumes:      - ./vector-setup/vector.yaml:/etc/vector/vector.yaml:ro      - /var/run/docker.sock:/var/run/docker.sock:ro    command: --config /etc/vector/vector.yaml  # Add any other services:  caddy:    image: caddy:latest    volumes:      - ./modules/caddy-setup/tmp/config:/config      - ./modules/caddy-setup/tmp/data:/data      - ./modules/caddy-setup/Caddyfile:/etc/caddy/Caddyfile    ports:      - "80:80"      - "443:443"

Since vector needs to communicate with the docker daemon we need to mount the docker socket and make it available to the vector container.

In the vector.yaml config, we can specify which bucket we want to funnel our logs to and which services we want to track:

data_dir: /var/lib/vectorsources:  docker_logs_source:    type: docker_logs    docker_host: "unix:///var/run/docker.sock"sinks:  s3_sink:    type: aws_s3    inputs: [docker_logs_source]    region: ap-south-1    bucket: my-service-logs    key_prefix: "container-logs/date=%Y-%m-%d"    compression: gzip    encoding:      codec: json

And that is all we need. Once we run docker compose up, after a while we should see our logs getting dumped into the S3 bucket.

This is a quick recipe to integrate it with CtrlP - my preferred file finder for vim.

In ~/.config/lvim/lua/treeutils.lua:

local api = require("nvim-tree.api")local M = {}-- Launch CtrlP from selected tree nodefunction M.launch_ctrlp()    local node = api.tree.get_node_under_cursor()    local is_folder = node.fs_stat and node.fs_stat.type == 'directory' or false    local basedir = is_folder and node.absolute_path or vim.fn.fnamemodify(node.absolute_path, ":h")    vim.cmd("CtrlP " .. basedir)endreturn M

In ~/.config/lvim/config.lua:

lvim.builtin.nvimtree.setup.on_attach = function(bufnr)    local api       = require "nvim-tree.api"    local treeutils = require "treeutils"    local function opts(desc)        return { desc = "nvim-tree: " .. desc, buffer = bufnr, noremap = true, silent = true, nowait = true }    end    api.config.mappings.default_on_attach(bufnr)    local useful_keys = {        ["<C-p>"] = { treeutils.launch_ctrlp, opts "Launch CtrlP" },        -- Other useful keybindings - cherry pick        ["l"] = { api.node.open.edit, opts "Open" },        ["o"] = { api.node.open.edit, opts "Open" },        ["<CR>"] = { api.node.open.edit, opts "Open" },        ["v"] = { api.node.open.vertical, opts "Open: Vertical Split" },        ["h"] = { api.node.navigate.parent_close, opts "Close Directory" },        ["C"] = { api.tree.change_root_to_node, opts "CD" },    }    require("lvim.keymappings").load_mode("n", useful_keys)end

Now, whenever we do a ctrl+p with our cursor on a directory node, we'll get a file selector scoped to that directory.