I recently worked on a good-old server-rendered web application written in Phoenix. One part of this app is a rather complex form that requires quite a bit of plumbing and UI logic that is IMO a bit more than what can be comfortably written and maintained in vanilla Javascript or even AlpineJS.

Long story short, I decided to embed a small JS app built with one of the popular SPA frameworks. There are many options for this such as React, or Vue, and many of their derivatives, but I decided to give Svelte a try since the way it works by compiling away the “framework” into runtime code should result in a smaller payload for my need to built a complex UI flow that is relatively limited in scope. In theory, at least. Plus, I’m also interested to checkout the framework to see how would fit in my toolbox.

The project uses Tailwind for the styling needs, and by default, Phoenix came with a webpack asset pipeline. After combing through dozens of pages of webpack config reference and github issues, I managed to setup a build that works well for both existing JS and CSS files for both the existing server-rendered pages and the SPA by setting up multiple entrypoints on webhook.

While it works for the most part, it is not a perfect setup. Because of the way webpack is integrated with Phoenix, every build requires a full page reload in order to see the changes. Webpack also needs to build the entire bundle every single time. All of that added up to a very slow feedback cycle. (I also wasn’t able to get the styles in the .svelte files to be bundled properly, but I’m sure that’s just me getting some part of the configs wrong).

Anyway, the entire build process with Webpack is just something that I’ve never been really happy with for various reasons, and one of the newer build-tool that I’ve been wanting to try is Vite. Vite is one of the more recent build tools that takes advantage of the availability of native ES module that is becoming more mainstream among browsers.

With ES modules, browsers can load individual modules without having to bundle them first. This means each module can be changed, rebuilt, and reloaded separately without affecting unrelated modules. In theory, that should cut down the feedback cycle signficantly as the dev server now does not have to rebuild the entire bundle on every single file change.

That also means it means Vite includes a dev server that is capable of serving individual transformed javascript modules (and also notify the browser of changes, etc). The backend integration with other frameworks often requires a dev server to be started separately in another port, which I find to be not very ergonomic, so I thought given Phoenix and Elixir’s capability to manage processes, it should be pretty trivial to start a proxy process so existence of the Vite server can be kinda opaque.

First thing first, remove Webpack and install Vite. This is what my package.json looks like:

{
  "repository": {},
  "description": " ",
  "license": "MIT",
  "type": "module",
  "scripts": {
    "watch": "vite",
    "preview": "vite preview",
    "build": "vite build",
    "check": "svelte-check --watch --diagnostic-sources 'js,svelte'"
  },
  "dependencies": {
    "phoenix": "file:../deps/phoenix",
    "phoenix_html": "file:../deps/phoenix_html"
  },
  "devDependencies": {
    "@sveltejs/vite-plugin-svelte": "^1.0.0-next.11",
    "@tsconfig/svelte": "^2.0.1",
    "autoprefixer": "^10.2.5",
    "date-fns": "^2.22.1",
    "postcss-load-config": "^3.1.0",
    "postcss-nested": "^5.0.5",
    "svelte": "^3.38.2",
    "svelte-check": "^2.2.3",
    "svelte-preprocess": "^4.7.3",
    "tailwindcss": "^2.1.2",
    "tslib": "^2.2.0",
    "typescript": "^4.3.2",
    "vite": "^2.4.0"
  }
}

My setup includes some additional dependencies as I like to use Typescript with svelte. Then, next thing to setup is to update the config/dev.exs file to setup the watcher.

  watchers: [
    npm: [
      "run",
      "watch",
      cd: Path.expand("../assets", __DIR__)
    ]
  ]

We also needs a vite.config.js file to tell Vite how to build.

import { defineConfig } from 'vite'
import { svelte } from '@sveltejs/vite-plugin-svelte'

export default defineConfig({
  plugins: [svelte()],
  publicDir: './static',
  build: {
    manifest:true,
    outDir: '../priv/static',
    rollupOptions: {
      input: {
        app: './js/app.js',
        fancyForm: './js/fancyForm/index.js',
      }
    }
  }
})

In the sample above, I defined 2 entry points, an app and fancyForm, each of them pointing to a javascript entry file. The app pointing to the global javascript bundle, and the fancyForm to the complex component that will only be used on certain pages. This is only used for the production build, during development, I will route the traffic to the files I need directly with the next part of the config.

Now, running mix phx.server will also start the Vite dev server, which would run on port 3000 by default. As the Phoenix dev server runs on port 4000 by default, I need to setup a proxy to forward the assets request to the vite dev server. There is probably a more lightweight solution that can be built with Plug, but as I wanted to have a proof of concept running quickly, I just searched hex.pm for a package that I can use, and I went with reverse_proxy_plug. Next to set it up in the app’s endpoint so it can redirect the appropriate traffic. I wanted an API similar to the Plug.Static, so I just made some minor changes around it.

defmodule VitePlug do

  @behaviour Plug
  @allowed_methods ~w(GET HEAD)

  alias Plug.Conn

  @impl true
  def init(opts) do
    %{
      only_rules: {
        Keyword.get(opts, :only, []) |> add_vite_paths(),
        Keyword.get(opts, :only_matching, [])
      },
      at: opts |> Keyword.fetch!(:at) |> Plug.Router.Utils.split()
    }
  end

  defp add_vite_paths(paths) do
    ["@vite", "__vite_ping", "node_modules", "@fs"] ++ paths
  end

  @impl true
  def call(conn = %Conn{method: meth}, %{at: at, only_rules: only_rules} = _options) when meth in @allowed_methods do
    segments = subset(at, conn.path_info)

    if allowed?(only_rules, segments) do
      opts = ReverseProxyPlug.init(upstream: "http://localhost:3000")
      ReverseProxyPlug.call(conn, opts)
      |> Conn.halt()
    else
      conn
    end
  end

  def call(conn, _options) do
    conn
  end

  defp subset([h | expected], [h | actual]), do: subset(expected, actual)
  defp subset([], actual), do: actual
  defp subset(_, _), do: []

  defp allowed?(_only_rules, []), do: false
  defp allowed?({[], []}, _list), do: true

  defp allowed?({full, prefix}, [h | _]) do
    h in full or (prefix != [] and match?({0, _}, :binary.match(h, prefix)))
  end
end

And it can be used as such:

if Mix.env() == :prod do
  plug Plug.Static,
    at: "/",
    from: :app,
    gzip: true,
    only: ~w(assets css fonts images favicon.ico robots.txt)
else
  plug VitePlug, at: "/", only: ~w(js css fonts images favicon.ico robots.txt)
end

It can certainly be improved further, but it will work for now. On production deployment, Vite will still be bundling our assets, so we will keep using Plug.Static for it, and for dev, the assets to the static assets will be proxied to the Vite server. In addition to the static paths, some vite-specific requests are also added as these are needed for things like HMR to work.

During development, the usual Routes.static_path helper can be used to generate the links to javascript entrypoints thats needed. As the request will be piped through the proxy server to the Vite server, and from there, additional imported dependencies such as CSS files or other modules will be done through ES modules loading, which will also be proxied to the Vite server. A script link to /@vite/client also needs to be included to enable various Vite dev server features.

Continuing on the example above, I’d want the app.js and app.css on every page of the app as thats used by various server-render pages, and also the /@vite/client as that is needed for Vite to work properly. I could update root layout/app.html.eex file to include the following:

<link rel="stylesheet" href="<%= Routes.static_path(@conn, "/css/app.css") %>"/>
<script type="module" src="<%= Routes.static_path(@conn, "/js/app.js") %>"></script>
<script type="module" src="<%= Routes.static_path(@conn, "/@vite/client") %>"></script>

So here Vite is acting as if it is a static file server, and I could request any file that I want as if it is a static file, and Vite will return the transformed javascript code. And because of the way the proxy is setup, I can keep using the static files as if they exist in the static folder and served by Plug.Static. Likewise, the fancyForm can be included with the snippet below. If the file imports any CSS file, there is no need to explicitly include the link tag to the CSS file, as the script will import the stylesheet as a module.

<script type="module" src="<%= Routes.static_path(@conn, "/js/fancyForm/index.js") %>"></script>

For production build tough, Vite will bundle the assets into chunks, and it will generate a manifest.json, that maps the entry points to their dependencies, and I’d need to include these chunks on the rendered HTML. A sample of the manifest file from Vite’s website looks like below:

{
  "main.js": {
    "file": "assets/main.4889e940.js",
    "src": "main.js",
    "isEntry": true,
    "dynamicImports": ["views/foo.js"],
    "css": ["assets/main.b82dbe22.css"],
    "assets": ["assets/asset.0ab0f9cd.png"]
  },
  "views/foo.js": {
    "file": "assets/foo.869aea0d.js",
    "src": "views/foo.js",
    "isDynamicEntry": true,
    "imports": ["_shared.83069a53.js"]
  },
  "_shared.83069a53.js": {
    "file": "assets/shared.83069a53.js"
  }
}

So now, I’d need to write something that takes an entry point, and then generate a static link to it and all its dependencies. While I’m at it, I decided to also write a generic function that can be used to include a static asset, and then generate the appropirate tags depending if it is running on prod or dev.

defmodule ViteLink do
  alias AppWeb.Router.Helpers, as: Routes

  defp is_prod(), do: Mix.env() == :prod

  # Generate vite client tags on dev, renders nothing on prod
  def client_tags(conn) do
    if is_prod() do
      nil
    else
      vite_client_src = Routes.static_path(conn, "/@vite/client")
      ~s(<script type="module" src="#{vite_client_src}"></script>)
      |> Phoenix.HTML.raw()
    end
  end

  # Generate single entry script tag to the entry point on dev,
  # generate tags for all its deps on prod
  def generate_tags(conn, entry_point) do
    if is_prod() do
      files = get_files(entry_point) |> Enum.reverse()
      get_links(conn, files)
    else
      get_tag(conn, {:js, "/#{entry_point}"})
      |> Phoenix.HTML.raw()
    end
  end

  def get_manifest() do
    manifest_file = Application.app_dir(:app, "priv/static/manifest.json")

    if File.exists?(manifest_file) do
      {
        :ok,
        File.read!(manifest_file)
        |> Jason.decode!()
      }
    else
      {:error, "manifest.json not found, pls run asset build"}
    end
  end

  def get_files(entry_point) do
    {:ok, manifest} = get_manifest()

    entry = Map.get(manifest, entry_point)

    files = [
      {:js, Map.get(entry, "file")} |
      Map.get(entry, "css")
      |> Enum.map(&({:css, &1}))
    ]

    process_file(
      Map.get(entry, "imports"),
      manifest,
      files
    )
  end

  def process_file(nil, _manifest, files), do: files
  def process_file([], _manifest, files), do: files
  def process_file([current | imports], manifest, files)do
    entry = Map.get(manifest, current)

    current_files = [
      {:js, Map.get(entry, "file")} |
      Map.get(entry, "css")
      |> Enum.map(&({:css, &1}))
    ]

    files = files ++ current_files

    process_file(
      imports,
      manifest,
      process_file(Map.get(entry, "imports"), manifest, files)
    )
  end

  def get_links(conn, files) do
    files
    |> Enum.map(fn {type, file} ->
      get_tag(conn, {type, "/#{file}"})
    end)
    |> Enum.join("\n")
    |> Phoenix.HTML.raw()
  end

  defp get_tag(conn, {:js, path}) do
    src = Routes.static_path(conn, path)
    ~s(<script type="module" src="#{src}"></script>)
  end

  defp get_tag(conn, {:css, path}) do
    src = Routes.static_path(conn, path)
    ~s(<link rel="stylesheet" href="#{src}"/>)
  end
end

With that, the tags from before can be rewritten as:

<%= ViteLink.client_tags(@conn) %>
<%= ViteLink.generate_tags(@conn, "js/app.js") %>

Now the links would work for both dev and production build. One thing to keep in mind though, is the Vite build outputs includes a digest in their filenames, and by default, Phoenix’s static_path would also include the file digest generated by mix phx.digest command. This means our call to Routes.static_path(conn, "assets/main.4889e940.js") would output something like assets/main.4889e940.<phoenix digest>.js.

This is not by itself a big issue as it simply have added another digest on top. A bit of wasted work, but doesn’t really break anything. The issue is when the Javascript code attempted to load a module, it would attempt to do so with only the Vite-digested file name. Since the generated script tags use the link with the phoenix digest, browsers would not recognize them as the same file, and would attempt to download the file again.

One option here is to simply not generate the dependencies script tag and let the browser load them via ESM, but this would introduce additional latency as browser would first need to download and parse the entry javascript files in order to know what are their dependencies.

I ended up disabling the phoenix digest entirely by removing the cache_static_manifest config key. This would cause Routes.static_path calls to not include the digest, so the file loaded by the script and the file included by the script tag to have the same name.

config :app, AppWeb.Endpoint,
  url: [host: "example.com", port: 80]
  # cache_static_manifest: "priv/static/cache_manifest.json" <-- Remove this line

I haven’t been able to find a documentation on what other behaviour the cache_static_manifest affects, and I’m rather occupied to do a source dive to find out, but so far it seems to have only affected the cache-busting digest.

And thats it. With the setup described above, we have a Phoenix project that would automatically starts a Vite webserver on mix phx.server, proxy the static files requests, and generates the correct static links for both dev and production run.

I thought of cleaning them up, maybe put in some fallback ports for the vite server, clean up some configs, fix some the runtime/compile time, maybe write some of the functions as compile-time macros, or even package them into a hex package, but I ended up not bothering with them as I’m not entirely happy with the setup.

While Vite indeed managed to deliver on its promise of super-fast feedback cycle while working on front-end code, the setup I’m working with is still inherently a server-rendered app. Given the way ES modules works, it means on every page load, the browser would need to reload all the dependencies again, which leads to a cascade of dependencies loading. This can be especially bad if the app has a deep dependency tree, where each page load can take close to 10 seconds to completely load all the client side dependencies.

Another issue I had, and this is specific to my particular setup, is that I prefer to use Typescript while working with Javascript. Vite uses esbuild to transform the typescript code into javascript, and esbuild does not perform any type of typechecking. This can be resolved by running another process like svelte-check that performs the typechecking separately, but I dont like the disconnect there.

I’d say Vite delivered on its promises. I kinda expected the issues on page-reload, but didn’t expect it to be so severe. I am happy with this current setup for this particular project, and definitely is an improvement over the config labyrinth that comes with Webpack, but I’m not keen on taking it further, hence I decided to write this post to at least document the approach. Also, few days after I got the setup working, I saw the tweet where it was announced that Phoenix will move to to esbuild, which IMO is definitely a good move, and makes having to deal with Webpack much less of an issue.

All in all, I’d say embedding an SPA inside a server-rendered app is an awkward place to be, and the setup above may work better if you are building an API with Phoenix and just wants a way to keep everything in a single repo. I find hard to give up the benefit of a server-rendered app, but when building a complex piece of UI, I feel vanilla javascript isn’t much of a solution either. Maybe one day someone will come-up with a client-side library built with server-side hydration as a first-class thought.