Build a set of projects with a single command

It would be useful to have a way to build a certain set of projects on a workspace.

Perhaps this can be achieved with global workspace targets, or via a filtering mechanism.

Followup to #10397

Any plans for this feature?

@filipesilva

Every project in a workspace already contains package.json (obviously). We could use peerDependencies to declare the dependencies between projects of the workspace. Then build and test commands would be able to detect the dependencies and build the libraries in the correct sequence.

I have written myself a node executable that does exactly that. Follows the dependency tree and builds the libraries in the correct sequence.

If that is something you see as a valuable addition to Angular CLI I would be more than happy to contribute.

@klemenoslaj

@klemenoslaj the approach we are looking at now would be able to do that implicitly, we hope. We're still finishing up the Architect API (this is what we use for build definitions).

@filipesilva

@filipesilva, that sounds good 👍

I however still like the idea of using peerDependencies, because it forces people to think about their dependencies (very important for published packages) and if that is done correctly, build sequence is automatically correct.

I can see how that can be a bit complex for someone, but I still think handling dependencies via package.json is a good way to go.

Are there any plans when it can be expected?

Is it possible with the new CLI builders announced here : https://blog.angular.io/introducing-cli-builders-d012d4489f1b ?

If yes could we have an example ?

Actually, the builder approach is just project related and not workspace related. We already implemented npm script that just analyzes all projects, builds dependency tree and calls sequentially build but still problems during watch mode: build is too slow!

During development time I guess only productive way is to change tsconfig path as described on #10643 (comment)

@meriturva

@meriturva Is this possible for multiple libraries within a single project? I understand schematics exposes options to invoke external schematics and builder is just an extension of schematics!?

@Goonersid our approach is just for a single project with multiple libraries. Right now we have all to test new build based on bazel before improving our internal solution.

@meriturva

@meriturva I'm looking for a cli cmd that builds all libraries within a single project... Is it possible with v6 or the latest stable? Something like ng build-all

@Goonersid you have to write you own task script and run as: npm run build-all

@Goonersid I was also looking for something like this and created a script, which reads the angular.json file and executes ng build for every project and every configuration. Also this script collects all failed builds and prints them to stdout at the end.

EDIT:
I am using nrwl/nx, so I don't have to build libraries manually.
So this script will probably not work for default Angular CLI.

The script looks like this:

import { ProjectType, WorkspaceSchema } from "@schematics/angular/utility/workspace-models";
import { execSync } from "child_process";
import { readFileSync } from "fs";

interface ExecParams {
  project: string;
  config: string;
}

interface ExecError extends ExecParams {
  message: string;
}

function buildAll() {
  const json: WorkspaceSchema = JSON.parse(readFileSync("./angular.json").toString());
  const errors: ExecError[] = Object.keys(json.projects)
    // Filter application-projects
    .filter(name => json.projects[name].projectType === ProjectType.Application)
    // Determine parameters needed for the build command
    .reduce<ExecParams[]>((arr, project) => {
      const proj = json.projects[project];
      let build = proj.architect && proj.architect.build;
      if (build) {
        arr = arr.concat(...Object.keys(build.configurations || {})
          .map(config => ({ project, config }))
        );
      }
      return arr;
    }, [])
    // Execute `ng build` and collect errors.
    .reduce<ExecError[]>((err, exec) => {
      try {
        console.log(`Building ${exec.project} (${exec.config}):`);
        execSync(`ng build --prod --project ${exec.project} --configuration ${exec.config}`, {stdio: "inherit"});
      }
      catch (error) {
        err.push({
          project: exec.project,
          config: exec.config,
          message: error.message
        });
      }
      console.log("\n");
      return err;
    }, []);

  // Conditionally log errors
  if (errors.length === 0)
    console.log("Completed");
  else {
    console.log("\n");
    errors.forEach(error => {
      console.error(`Building ${error.project} (${error.config}) failed:\n\t${error.message}`);
    });
  }
}

buildAll();

This script can be compiled to Javascript and then you can run it using NodeJs.

@Springrbua but you're not following any dependencies, therefore if some library depends on another this will break or work by accident.

@klemenoslaj

@klemenoslaj You are right, I never used the normal Angular CLI for multi app projects and therefore forgot, that you need to build libraries yourself.
I am using nrwl/nx, which automatically builds all needed libraries when executing ng build, so that's not a problem in my case.

@klemenoslaj

@klemenoslaj the approach we are looking at now would be able to do that implicitly, we hope. We're still finishing up the Architect API (this is what we use for build definitions).

There is not much going on in this feature request. I would still like to offer my repository for possible inspiration but will leave it at that due to lack of interest.

@filipesilva feel free to have a peek at the following repository. I implemented an implicit approach, so no additional config is required whatsoever.

The repository: https://github.com/klemenoslaj/ong-cli

This is indeed looks like a rough edge where the community seem to be struggling. I was able to stumble upon a lot of SO threads pointing to the need of this feature, for years.

To point out a few:

and all of these threads seem to be very genuine.

It would be very great when we run serve/build, cli would automatically build its depended libraries as well. The solution should also be compatible when ng serving multiple apps.

As of now what I'm doing is open a terminal per library and run ng build --watch. Guess how many terminal tabs I'm keeping!

I have to move nx to have this: https://nx.dev/generators/workspace-generators
I can build a set of project easy, example

import { appAndLibSetting } from '../../new-tool/common-setting.tool';
import { Tree, formatFiles, readProjectConfiguration } from '@nrwl/devkit';
import { wrapAngularDevkitSchematic } from '@nrwl/devkit/ngcli-adapter';
import shellGenerator from '../shell';
import sharedGenerator from '../shared';

export default async function (tree: Tree, schema: any) {
  const domainName = `${schema.directory}-${schema.device}`;
  await createApplication(tree, schema);
  deleteUnecessaryFile(tree, schema, domainName);
  await createDependanceLibrary(tree, schema, domainName);

  await formatFiles(tree);
  return () => {};
}

function deleteUnecessaryFile(tree: Tree, schema: any, domainName: string) {
  const libraryRoot = readProjectConfiguration(tree, domainName).root;
  tree.delete(`${libraryRoot}/assets/.gitkeep`);
  tree.delete(`${libraryRoot}/environments/environment.prod.ts`);
  tree.delete(`${libraryRoot}/environments/environment.ts`);
}

async function createApplication(tree: Tree, schema: any) {
  const libraryGenerator = wrapAngularDevkitSchematic('@nrwl/angular', 'app');
  await libraryGenerator(tree, {
    ...schema,
    ...appAndLibSetting,
    name: schema.device,
    directory: schema.directory,
    routing: false,
    tags: `scope:shared,type:app`,
  });
}

async function createDependanceLibrary(tree: Tree, schema: any, domainName: string) {
  await shellGenerator(tree, {
    directory: `${schema.directory}/${schema.device}`,
  });
  await sharedGenerator(tree, {
    directory: `${schema.directory}/${schema.device}`,
  });
}

import { Tree, formatFiles, readProjectConfiguration, generateFiles, joinPathFragments, readJson } from '@nrwl/devkit';
import { wrapAngularDevkitSchematic } from '@nrwl/devkit/ngcli-adapter';
import { classify, dasherize } from '@nrwl/workspace/src/utils/strings';
import { appAndLibSetting } from '../../new-tool/common-setting.tool';

export default async function (tree: Tree, schema: any) {
  await createShellLibrary(tree, schema, 'shell');
  const projectName = dasherize(`${schema.directory}-shell`).replace(/\//g, '-');
  addShellModule(tree, schema, projectName, 'shell');
  await formatFiles(tree);
  return () => {};
}

export async function createShellLibrary(tree: Tree, schema: any, type) {
  const libraryGenerator = wrapAngularDevkitSchematic('@nrwl/angular', 'lib');
  await libraryGenerator(tree, {
    name: `${schema.directory}/${type}`,
    tags: `scope:${type},scope:shared,type:${type}`,
    ...appAndLibSetting,
  });
}

export function addShellModule(tree: Tree, schema: any, projectName, type) {
  const libraryRoot = readProjectConfiguration(tree, projectName).root;
  generateFiles(tree, joinPathFragments(__dirname, `./files`), `${libraryRoot}/src/lib/${dasherize(`${schema.directory}-${type}`)}`, {
    ...schema,
    sharedModule: classify(`${schema.directory}-shared-module`.replace(/\//g, '-')),
    shellModule: classify(`${schema.directory}-shell-module`.replace(/\//g, '-')),
    shellName: dasherize(`${schema.directory}-shell`.replace(/\//g, '-')),
    sharedName: dasherize(`${schema.directory}-shared`.replace(/\//g, '-')),
  });
}

Our way (without using nx) is to analyze all projects from angular.json file and create a dependency tree, then run multiple build commands based on tree (parallelly) to speed up the process.

See how we read angular.json file: https://medium.com/we-code/angular-inspect-workspace-programmatically-3cdfeb9b93d4

Our way (without using nx) is to analyze all projects from angular.json file and create a dependency tree, then run multiple build commands based on tree (parallelly) to speed up the process.

See how we read angular.json file: https://medium.com/we-code/angular-inspect-workspace-programmatically-3cdfeb9b93d4

Hi,

We are facing the same problem as you and you're solution sounds perfect! Would you be willing to share how you create the dependency tree and run build commands based on it?

Thanks in advance

@meriturva

Our way (without using nx) is to analyze all projects from angular.json file and create a dependency tree, then run multiple build commands based on tree (parallelly) to speed up the process.
See how we read angular.json file: https://medium.com/we-code/angular-inspect-workspace-programmatically-3cdfeb9b93d4

Hi,

We are facing the same problem as you and you're solution sounds perfect! Would you be willing to share how you create the dependancy tree and run build commands based on it?

Thanks in advance

@meriturva I would also be interested in how you did this because it sounds very similar to what we did :)

@K3CharlieStuart current solutions I know now are:

either take: https://github.com/klemenoslaj/ong-cli // download and build the repo, than you can link it as npm dependency
or check how a dependancy tree is created and build commands are run // but this is not working in parallelly
- https://github.com/dlr-eoc/ukis-frontend-libraries/tree/main/scripts/library
- https://github.com/dlr-eoc/ukis-frontend-libraries/blob/0e5949ed4040a605de40a899be18e04c7d84b73d/scripts/library/index.ts#L100
use nx ...

@boeckMt

@boeckMt thanks for the suggestions!

I'm just looking into your second suggestion but I cant seem to figure out how we would utilise this in our project. Would it be a case of copying the script folder into our application and somehow executing the quoted function?

Here is just a code snippet of how we build our dependencies tree, it is really simple,
the complex one is to run all scripts parallel and that script it is on our private repo right now and I cannot share it for now.
May be I could arrange an article (or a few) about that. Would you be interested?

export function buildPeerDependenciesTree(filterArchitect: string): Record<string, PackageNode> {

  const angularJson = JSON.parse(fs.readFileSync('angular.json', 'utf8'));

  const nodes: Record<string, PackageNode> = {}; 

  Object.values(angularJson.projects).forEach((value) => {
    if ((value as any).projectType === 'library' && (value as any).architect[filterArchitect]) {
      const packageContent = JSON.parse(fs.readFileSync(path.join((value as any).root, 'package.json'), 'utf8'));
      const moduleName = packageContent.name;
      const peerDependencies = new Set<string>();

      // Look for peerDependencies also in package.json in subFolders
      const allPackageJson = glob.sync(`${(value as any).root}/**/package.json`);
      allPackageJson.forEach(filePath => {
        const fileContent = JSON.parse(fs.readFileSync(filePath, 'utf8'));
        if (fileContent && fileContent.peerDependencies) {
          Object.keys(fileContent?.peerDependencies).forEach(peerDependency => peerDependencies.add(peerDependency));
        }
      });

      if (!nodes[moduleName]) {
        nodes[moduleName] = new PackageNode(moduleName, (value as any).root, (value as any).sourceRoot);
      }

      nodes[moduleName].package = packageContent;

      peerDependencies.forEach((nodeDependency) => {
        // Check if dependency is present locally (to be compile)
        if (angularJson.projects[nodeDependency]?.architect[filterArchitect]) {
          if (!nodes[nodeDependency]) {
            nodes[nodeDependency] = new PackageNode(nodeDependency, (angularJson.projects[nodeDependency] as any).root, (angularJson.projects[nodeDependency] as any).sourceRoot);
          }

          nodes[moduleName].originalDependencies.push(nodeDependency);
          nodes[moduleName].dependencies.push(nodeDependency);
        }
      });
    }
  });

  return nodes;
}

@boeckMt

@boeckMt thanks for the suggestions!

I'm just looking into your second suggestion but I cant seem to figure out how we would utilise this in our project. Would it be a case of copying the script folder into our application and somehow executing the quoted function?

Sorry for repeating the same thing, but I feel we are going back to basics here - reading angular.json etc.

Have a look at the #11002 (comment) where I made an entire repo public - it even supports parallel builds no matter the complexity of dependency tree.

Running example in the following comment: #11002 (comment)

Due to lack of interest I did not yet made this an npm package tho, but feel free to consume & contribute.

Definitely interested in something like this. It would be nice to specify what projects to build and the order to build them. I know it can be done manually through scripts in the package.json, but doing so is cumbersome and inconsistent with the highly configurable nature of the build process in general. Also, a first-party solution from Angular would help streamline and standardize the process. It looks like this isn't being pursued at the moment in a first-party solution, so I'm adding my comment here to give this more support.

I would be highly interested in a "native" Angular CLI solution as well.
It would greatly improve our ability to guarantee development quality and provide a consistent dev environment to our developers.
Providing hints at https://angular.dev/tools/libraries would have been nice too.

Navigation Menu

Build a set of projects with a single command #11002

17 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Build a set of projects with a single command #11002

Description

Activity

abhijeetkpawar commented on Jan 7, 2019

klemenoslaj commented on Feb 20, 2019

filipesilva commented on Feb 21, 2019

klemenoslaj commented on Feb 21, 2019

RobinBomkampDv commented on Apr 25, 2019

jpleclerc commented on Apr 26, 2019

meriturva commented on Apr 26, 2019

siddharth1903 commented on Aug 13, 2019

meriturva commented on Aug 13, 2019

siddharth1903 commented on Aug 13, 2019

meriturva commented on Aug 13, 2019

probert94 commented on Aug 28, 2019

klemenoslaj commented on Sep 10, 2019

probert94 commented on Sep 10, 2019

17 remaining items

klemenoslaj commented on Mar 22, 2022

vajahath commented on May 5, 2022

hiepxanh commented on May 5, 2022

meriturva commented on May 5, 2022

CharlieStuart1995 commented on May 19, 2022

boeckMt commented on May 19, 2022

CharlieStuart1995 commented on May 19, 2022

meriturva commented on May 19, 2022

klemenoslaj commented on May 19, 2022

PrestonMonteWest commented on Jun 11, 2024

Volvohorst commented on Oct 16, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions