# Optimize large bundles' build size with content source compilation dependency duplication

Source: https://dev.arcxp.com/pagebuilder-engine/how-to-guides/optimization-and-deployment/optimize-large-bundles-build-size-with-content-source-compilation-dependency-duplication/

_This feature was introduced in Engine 4.1.8 and 5.0.4 releases that brings additional compiler options using a new `arc.config.json` configuration file. See [arc.config.json documentation](https://dev.arcxp.com/pagebuilder-engine/how-to-guides/optimization-and-deployment/arcconfigjson) to learn more about other compiler options._

## Introduction and Demo

[ Play ](https://youtube.com/watch?v=SpYVGBMsDnE) 

Optimize large bundles' build size for Lambdas

Learn how to optimize build sizes to avoid deployment failures due to Lambda file limitations

When Engine compiles your bundle source code into Lambda-ready code, it produces individual server-side mini bundles for each of your content sources so that when a `/pf/api/content` call is received, a leaner Lambda handler can be executed to fetch content. This process produces unnecessary code duplication when many content sources import the same dependencies. Depending on the number of content sources and imported dependencies, this caused the compiled server-side code to be bloated and resulted in deployment failures due to exceeding the AWS Lambda file size limitations.

This new feature utilizes `webpack.externals` to minimize the duplication of dependencies inside the Lambda after compilation by allowing the exclusion of `node_modules` to be removed from content sources. This is possible because content sources are invoked server-side.

Two new variables are exposed to engine through an optional file in the root of your bundle called `arc.config.json`:

- `compiler.contentSources.excludeModules` is essentially a boolean flag for now, but allowing `*` opens the possibility to support file globs in the future. When this property is set to `*`, it will exclude all dependencies in `node_modules` from being bundled into the content source.
- `compiler.contentSources.includeModules` allows developers to explicitly bundle specific modules into the content source when this functionality is enabled. This allows customers to retain some backwards compatibility when opting into this. In this example, we are showcasing a common logging library for typescript (`typescript-logging`) that has side effects when included from `node_modules`. If importing a dependency from `node_modules` affects the behavior of your content source, you may want to designate the dependency to be bundled directly into your content source to maintain the existing behavior.
1. Create or update the `arc.config.json` in the root of your bundle to have the following properties:  
```json
{
  "compiler": {
      "contentSources": {
          "excludeModules": "*",
          "includeModules": ["typescript-logging"]
      }
  }
}
```
2. Re-run the bundle and check the file sizes for `.fusion/build/content/sources`  
> **Note:**  
>  
> This capability does not change the client-side bundle that gets downloaded and used by the browser at render time, so it has no impact to your readers experience. This module de-duplication only affects the content source that is executed on the server.

## Benchmarks

We have tested a few bundles that exceed lambda size limits. After evaluating different build artifacts, we found that compiled content sources were the main culprit for deployment issues on the AWS Lambda filesystem.

By enabling this configuration, the PageBuilder compiler will exclude `node_modules` from bundling into the content sources. In this example, the majority of content sources were importing the same dependency. The actual JavaScript in the `content/sources` folder was quite optimized and DRY (don’t repeat yourself), but 1MB of dependencies multiplied by 100 content sources would previously generate 99MB of duplicated code. By removing this duplication during compilation, this yields a substantial file size reduction.

We simply enabled build configuration in the example above, and checked the total build folder size using the `du -h -d 2 .fusion/build` command, that just calculates the directory sizes starting from `.fusion/build` directory down to 2 levels deep.

Here are some examples we ran in our benchmarks:

### Bundle1 - 77% size reduction

| Before:▶ du -h -d 2 .fusion/build 101M	build/content/sources 101M	build/content 476K	build/components/output-types 652K	build/components 16K	build/environment 102M	build | After:▶ du -h -d 2 .fusion/build 23M	build/content/sources 23M	build/content 476K	build/components/output-types 652K	build/components 16K	build/environment 24M	build |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

### Bundle2 - 65% size reduction

| Before:▶ du -h -d 2 .fusion/build 336M    build/content/sources 336M    build/content 6.0M    build/components/output-types 6.2M    build/components 12K    build/environment 343M    build | After:▶ du -h -d 2 .fusion/build 79M    build/content/sources 79M    build/content 6.0M    build/components/output-types 6.2M    build/components 12K    build/environment 86M    build |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

### Bundle 3 - 39% size reduction

| Before:▶ du -h -d 2 .fusion/build 44K    build/content/schemas 119M    build/content/sources 119M    build/content 19M    build/components/output-types 20M    build/components 28K    build/environment 140M    build | After:▶ du -h -d 2 .fusion/build 44K    build/content/schemas 66M    build/content/sources 66M    build/content 19M    build/components/output-types 20M    build/components 28K    build/environment 86M    build |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |

### Additional context about size limits

AWS Lambda enforces a 250MB total size limit that is shared between PageBuilder Engine core, Arc XP Blocks (if used), and the custom bundle code.