Refactor npm scripts, clean up unused ones (#1392)

Co-Authored-By: Artur Paikin <artur@arturpaikin.com>

.github/CONTRIBUTING.md

@@ -23,7 +23,7 @@ Unit tests are using Jest and can be run with:
 npm run test:unit
 ```
 
-For acceptance (or end-to-end) tests, we use [Webdriverio](http://webdriver.io). For it to run locally, you need to install a Selenium standalone server. Just follow [the guide](http://webdriver.io/guide.html) to do so. You can also install a Selenium standalone server from NPM:
+For end-to-end tests, we use [Webdriverio](http://webdriver.io). For it to run locally, you need to install a Selenium standalone server. Just follow [the guide](http://webdriver.io/guide.html) to do so. You can also install a Selenium standalone server from NPM:
 
 ```bash
 npm install selenium-standalone -g
@@ -39,13 +39,13 @@ selenium-standalone start
 After you have installed and launched the selenium standalone server, run:
 
 ```bash
-npm run test:acceptance:local
+npm run test:endtoend:local
 ```
 
-By default, `test:acceptance:local` uses Firefox. You can use a different browser, like Chrome, by passing the `-b` flag:
+By default, `test:endtoend:local` uses Firefox. You can use a different browser, like Chrome, by passing the `-b` flag:
 
 ```bash
-npm run test:acceptance:local -- -b chrome
+npm run test:endtoend:local -- -b chrome
 ```
 
 > Note: The `--` is important, it tells npm that the remaining arguments should be interpreted by the script itself, not by npm.
@@ -53,13 +53,13 @@ npm run test:acceptance:local -- -b chrome
 You can run in multiple browsers by passing multiple `-b` flags:
 
 ```bash
-npm run test:acceptance:local -- -b chrome -b firefox
+npm run test:endtoend:local -- -b chrome -b firefox
 ```
 
 When trying to get a specific integration test to pass, it's not that helpful to continuously run _all_ tests. You can use the `--suite` flag to run tests from a single `./test/endtoend` folder. For example, `--suite thumbnails` will only run the tests from `./test/endtoend/thumbnails`. Of course, it can also be combined with one or more `-b` flags.
 
 ```bash
-npm run test:acceptance:local -- -b chrome --suite thumbnails
+npm run test:endtoend:local -- -b chrome --suite thumbnails
 ```
 
 These tests are also run automatically on Travis builds with [SauceLabs](https://saucelabs.com/) cloud service using different OSes.

.travis.yml

@@ -9,7 +9,7 @@ install:
 script:
 - npm run build
 - npm run test
-- if [ "${TRAVIS_PULL_REQUEST}" = "false" ]; then npm run test:acceptance; fi
+- if [ "${TRAVIS_PULL_REQUEST}" = "false" ]; then npm run test:endtoend; fi
 - npm run uploadcdn
 cache:
   apt: true

CHANGELOG.md

@@ -95,7 +95,6 @@ What we need to do to release Uppy 1.0
 
 - [ ] chore: hunt down all `@TODO`s and either fix, or remove, or move to github issues/changelog backlog
 - [ ] chore: remove dead code/commented blocks
-- [ ] chore: remove the not-working npm scripts
 - [ ] chore: rewrite all code based on prettier+standardjs.com
 - [ ] locale: update the locales of languages that we know ourselves. leave rest to community
 - [ ] locale: cdn (just in folder like Robodog, will attach to global) / for module to all languages in one big `@uppy/locales`
@@ -109,7 +108,8 @@ What we need to do to release Uppy 1.0
 - [ ] transloadit: add error reporting, see https://github.com/transloadit/jquery-sdk/blob/891e99b08dd8142d8d8adc0553e6511967635ad7/js/lib/Modal.js#L122-L136 (@goto-bus-stop, @arturi)
 - [ ] transloadit: should adhere cancel event and abort assembly (@arturi, @goto-bus-stop)
 - [ ] dashboard: optional alert `onbeforeunload` while upload is in progress, safeguarding from accidentaly navigating away from a page with an ongoing upload
-
+- [x] chore: remove the not-working npm scripts (@kvz, @arturi)
+- [x] build: (BREAKING) `npm run dev` no longer starts Companion by default, use `npm run dev:with-companion` for that (@arturi)
 - [x] core: uppy should not crash or be slow for many files. Specifically: be able to drop 5 files (or 7mb) without the upload button to take 2 seconds to appear
 - [x] uppy-server: bump minor and deprecate that on npm in favour of @uppy/companion (@arturi)
 - [x] dashboard: implement Alex and Artur’s Dashboard redesign (@arturi)

bin/bootandkill-servers

@@ -1,100 +0,0 @@
-#!/usr/bin/env bash
-# How to run:
-#
-#  ./bootandkill-servers ./test-acceptance
-#
-# this will boot hexo & companion, run the script that you provided as an argument,
-# and tear down the servers
-
-set -o pipefail
-set -o errexit
-set -o nounset
-set -o xtrace
-
-# Set magic variables for current file & dir
-__dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-__file="${__dir}/$(basename "${BASH_SOURCE[0]}")"
-__base="$(basename ${__file} .sh)"
-__root="$(cd "$(dirname "${__dir}")" && pwd)"
-
-if [ ! -f "${__root}/env.sh" ]; then
-  cp "${__root}/env.example.sh" "${__root}/env.sh"
-fi
-if [ "${COMPANION_DROPBOX_KEY:-}" = "" ] || [ "${COMPANION_DROPBOX_KEY:-***}" = "***" ]; then
-  source "${__root}/env.sh"
-fi
-if [ "${COMPANION_DROPBOX_KEY:-}" = "" ] || [ "${COMPANION_DROPBOX_KEY:-***}" = "***" ]; then
-  echo "[${__base}] Env var COMPANION_DROPBOX_KEY still had the example value '${COMPANION_DROPBOX_KEY:-}'. "
-  echo "[${__base}] Please save the actual secrets in '${__root}/env.sh' and try again"
-  exit 1
-fi
-
-function killProcessListeningOnPort () {
-  local port="${1}"
-  lsof -n -i4TCP:${port} | awk '/LISTEN/ {print $2}' |xargs --no-run-if-empty kill -9 || true
-}
-
-function cleanup_servers () {
-  echo "[${__base}] --> Killing any server listening on port 4000"
-  killProcessListeningOnPort 4000 || true
-
-  echo "[${__base}] --> Killing any server listening on port 3020"
-  killProcessListeningOnPort 3020 || true
-
-  kill -9 ${tailPid}
-}
-
-function waitForPortOpen () {
-  local port="${1}"
-  local limit="${2:-100}"
-  local attempts=0
-  echo "[${__base}] waiting on port ${port} to open... "
-  while ! echo exit | nc localhost ${port}; do
-    let "attempts = attempts + 1"
-    echo "[${__base}] still waiting on port ${port} to open... (${attempts} / ${limit}) "
-    sleep 1
-    if [ "${attempts}" -ge "${limit}" ]; then
-      echo "[${__base}] --> Port did not open for ${limit} seconds. Aborting. "
-      exit 1
-    fi
-  done
-}
-
-echo "[${__base}] --> Killing any server listening on port 4000"
-killProcessListeningOnPort 4000 || true
-echo "[${__base}] --> Killing any server listening on port 3020"
-killProcessListeningOnPort 3020 || true
-
-echo "[${__base}] --> Start webserver and companion in the background"
-rm -f nohup.out || true
-touch nohup.out
-nohup npm run test:serve &
-tail -f nohup.out &
-tailPid=${!}
-
-trap cleanup_servers EXIT
-
-
-
-echo "[${__base}] --> Wait for hexo webserver to be online"
-waitForPortOpen 4000
-
-echo "[${__base}] --> Wait for companion to be online"
-waitForPortOpen 3020
-
-echo "[${__base}] --> Running acceptance tests"
-
-
-buildWait=20
-if [ "${TRAVIS:-}" = "true" ]; then
-  buildWait=60
-  echo "Increasing build wait because Travis has no Hexo build cache"
-fi
-
-echo "[${__base}] --> Sleeping ${buildWait}s, because the port may be open, but Hexo may still be injecting/building stuff"
-echo "[${__base}] --> and I don't know yet how to check for that"
-sleep ${buildWait}
-
-# @todo: Instead of this `sleep` we can wait until the required files are in public : )
-
-${@}

bin/gzip.js

@@ -1,42 +0,0 @@
-var path = require('path')
-var chalk = require('chalk')
-var glob = require('glob')
-var exec = require('child_process').exec
-
-function handleErr (err) {
-  console.error(chalk.red('✗ Error:'), chalk.red(err.message))
-}
-
-function gzip (file) {
-  return new Promise(function (resolve, reject) {
-    var fileName = path.basename(file)
-    var gzipCommand = 'gzip < ' + file + ' > ' + file + '.gz'
-    exec(gzipCommand, function (error, stdout, stderr) {
-      if (error) {
-        handleErr(error)
-        reject(error)
-        return
-      }
-      console.info(chalk.green('✓ Gzipped: '), chalk.magenta(fileName + '.gz'))
-      resolve()
-    })
-  })
-}
-
-function gzipDist () {
-  return new Promise(function (resolve) {
-    glob('./packages/uppy/dist/**/*.*(css|js)', function (err, files) {
-      if (err) console.log(err)
-      var gzipPromises = []
-      files.forEach(function (file) {
-        gzipPromises.push(gzip(file))
-      })
-      return Promise.all(gzipPromises).then(function () {
-        console.info(chalk.yellow('✓ Gzipped everything yo 🎉'))
-        resolve()
-      })
-    })
-  })
-}
-
-gzipDist()

bin/release

@@ -58,7 +58,7 @@ git add README.md
 # Add readme file to the main `uppy` package.
 cp README.md packages/uppy/README.md
 
-npm run clean
+npm run build:clean
 FRESH=1 npm run build
 
 git commit --allow-empty -m "Release"

examples/dev/package.json

@@ -6,7 +6,7 @@
     "watchify": "^3.11.0"
   },
   "scripts": {
-    "watch": "watchify -t babelify -t aliasify main.js -o bundle.js -vd"
+    "watch:sandbox": "watchify -t babelify -t aliasify main.js -o bundle.js -vd"
   },
   "aliasify": {
     "aliases": {

package.json

@@ -81,58 +81,56 @@
   },
   "scripts": {
     "bootstrap": "lerna bootstrap",
-    "build:bundle": "node ./bin/build-js.js",
+    "build:bundle": "node ./bin/build-bundle.js",
+    "build:clean": "rm -rf packages/*/lib packages/@uppy/*/lib packages/*/dist packages/@uppy/*/dist",
     "build:companion": "cd ./packages/@uppy/companion && npm run build",
     "build:css": "node ./bin/build-css.js",
-    "build:gzip": "node ./bin/gzip.js",
     "build:js": "npm-run-all build:lib build:bundle",
     "build:lib": "node ./bin/build-lib.js",
-    "build": "npm-run-all --parallel build:js build:css build:companion --serial build:gzip size",
-    "clean": "rm -rf packages/*/lib packages/@uppy/*/lib packages/*/dist packages/@uppy/*/dist",
-    "contributors:save": "replace-x -m '<!--contributors-->[\\s\\S]+<!--/contributors-->' \"<!--contributors-->\n## Contributors\n\n$(npm run --silent contributors)\n<!--/contributors-->\" README.md",
-    "contributors": "githubcontrib --owner transloadit --repo uppy --cols 6 $([ \"${GITHUB_TOKEN:-}\" == \"\" ] && echo \"\" || echo \"--authToken ${GITHUB_TOKEN}\") --showlogin true --sortOrder desc",
+    "build": "npm-run-all --parallel build:js build:css build:companion --serial size",
+    "contributors:fetch": "githubcontrib --owner transloadit --repo uppy --cols 6 $([ \"${GITHUB_TOKEN:-}\" == \"\" ] && echo \"\" || echo \"--authToken ${GITHUB_TOKEN}\") --showlogin true --sortOrder desc",
+    "contributors:save": "replace-x -m '<!--contributors-->[\\s\\S]+<!--/contributors-->' \"<!--contributors-->\n## Contributors\n\n$(npm run --silent contributors:fetch)\n<!--/contributors-->\" README.md",
     "dev:browsersync": "browser-sync start --no-open --no-ghost-mode false --server examples/dev --port 3452 --serveStatic packages/uppy/dist --files \"examples/dev/bundle.js, packages/uppy/dist/uppy.min.css, packages/uppy/lib/**/*\"",
-    "dev:js": "cd examples/dev && npm run watch",
-    "dev:no-companion": "npm-run-all --parallel dev:js watch:css watch:js:lib dev:browsersync",
-    "dev": "npm-run-all --parallel start:companion dev:js watch:css watch:js:lib dev:browsersync",
+    "dev:watch-sandbox": "cd examples/dev && npm run watch:sandbox",
+    "dev:with-companion": "npm-run-all --parallel start:companion dev:watch-sandbox watch dev:browsersync",
+    "dev": "npm-run-all --parallel dev:watch-sandbox watch dev:browsersync",
     "lint:fix": "npm run lint -- --fix",
     "lint:staged": "lint-staged",
     "lint": "eslint . --cache",
     "release": "./bin/release",
     "size": "echo 'JS Bundle mingz:' && cat ./packages/uppy/dist/uppy.min.js | gzip | wc -c && echo 'CSS Bundle mingz:' && cat ./packages/uppy/dist/uppy.min.css | gzip | wc -c",
-    "start:companion": "sh ./bin/companion",
-    "start": "npm-run-all --parallel watch start:companion web:preview",
-    "test:acceptance:local": "npm run test:build && wdio test/endtoend/wdio.local.conf.js",
-    "test:acceptance": "npm run test:prepare-ci && wdio test/endtoend/wdio.remote.conf.js",
-    "test:build-ci": "./bin/endtoend-build-ci",
-    "test:build": "./bin/endtoend-build",
+    "start:companion": "./bin/companion",
+    "start": "npm-run-all --parallel watch start:companion web:start",
     "test:companion": "cd ./packages/@uppy/companion && npm run test",
-    "test:prepare-ci": "npm-run-all --parallel --race test:registry test:build-ci",
-    "test:registry": "verdaccio --listen 4002 --config test/endtoend/verdaccio.yaml",
+    "test:endtoend:build-ci": "./bin/endtoend-build-ci",
+    "test:endtoend:build": "./bin/endtoend-build",
+    "test:endtoend:local": "npm run test:endtoend:build && wdio test/endtoend/wdio.local.conf.js",
+    "test:endtoend:prepare-ci": "npm-run-all --parallel --race test:endtoend:registry test:endtoend:build-ci",
+    "test:endtoend:registry": "verdaccio --listen 4002 --config test/endtoend/verdaccio.yaml",
+    "test:endtoend": "npm run test:endtoend:prepare-ci && wdio test/endtoend/wdio.remote.conf.js",
     "test:type": "tsc -p .",
     "test:unit": "npm run build:lib && jest",
     "test:watch": "jest --watch",
     "test": "npm-run-all lint test:unit test:type test:companion",
-    "travis:deletecache": "travis cache --delete",
     "uploadcdn": "bin/upload-to-cdn.sh",
     "version": "./bin/sync-version-numbers",
     "watch:css": "onchange 'packages/**/*.scss' --initial --verbose -- npm run build:css",
+    "watch:js:bundle": "onchange 'packages/{@uppy/,}*/src/**/*.js' --initial --verbose -- npm run build:bundle",
     "watch:js:lib": "onchange 'packages/{@uppy/,}*/src/**/*.js' --initial --verbose -- npm run build:lib",
-    "watch:js": "onchange 'packages/{@uppy/,}*/src/**/*.js' --initial --verbose -- npm run build:bundle",
-    "watch": "npm-run-all --parallel watch:js watch:css",
-    "web:build": "npm-run-all web:inject web:generate web:examples",
-    "web:bundle:update:watch": "onchange 'packages/uppy/dist/**/*.css' 'packages/uppy/dist/**/*.js' --initial --verbose -- npm run web:inject",
+    "watch": "npm-run-all --parallel watch:*",
+    "web:build-examples": "cd website && node build-examples.js",
+    "web:build": "npm-run-all web:inject-bundles-misc web:generate web:build-examples web:inject-frontpagecodesample",
+    "web:bundle-watch-inject": "onchange 'packages/uppy/dist/**/*.css' 'packages/uppy/dist/**/*.js' --initial --verbose -- npm run web:inject-bundles-misc",
     "web:clean": "cd website && touch db.json && ./node_modules/.bin/hexo clean",
-    "web:deploy": "npm-run-all web:clean web:disc web:build && ./bin/web-deploy",
-    "web:disc": "node ./bin/disc.js",
-    "web:examples:watch": "cd website && node build-examples.js watch",
-    "web:examples": "cd website && node build-examples.js",
+    "web:deploy": "npm-run-all web:clean web:inject-disc web:build && ./bin/web-deploy",
     "web:generate": "cd website && touch db.json && ./node_modules/.bin/hexo generate",
-    "web:inject:frontpagecodesample": "npm run web:generate && cp -f public/frontpage-code-sample.html ./themes/uppy/layout/partials/frontpage-code-sample.html",
-    "web:inject": "cd website && node inject.js",
+    "web:inject-bundles-misc": "cd website && node inject.js",
+    "web:inject-disc": "node ./bin/disc.js",
+    "web:inject-frontpagecodesample": "npm run web:generate && cp -f website/public/frontpage-code-sample.html website/themes/uppy/layout/partials/frontpage-code-sample.html && touch website/themes/uppy/layout/index.ejs",
     "web:install": "cd website && npm install",
-    "web:preview": "npm-run-all build:lib --parallel web:examples:watch web:bundle:update:watch web:serve",
-    "web:serve": "cd website && touch db.json &&./node_modules/.bin/hexo server"
+    "web:start": "npm-run-all build:lib --parallel web:watch-examples web:bundle-watch-inject web:watch",
+    "web:watch-examples": "cd website && node build-examples.js watch",
+    "web:watch": "cd website && touch db.json &&./node_modules/.bin/hexo server"
   },
   "jest": {
     "automock": false,

test/endtoend/wdio.local.conf.js

@@ -1,8 +1,8 @@
 const base = require('./wdio.base.conf')
 const { CompanionService } = require('./utils')
 
-// Use "npm run test:acceptance:local -- -b chrome" to test in chrome
-// "npm run test:acceptance:local -- -b firefox -b chrome" to test in FF and chrome
+// Use "npm run test:endtoend:local -- -b chrome" to test in chrome
+// "npm run test:endtoend:local -- -b firefox -b chrome" to test in FF and chrome
 let prevIsDashB = false
 const capabilities = []
 process.argv.forEach((arg) => {

website/inject.js

@@ -17,7 +17,7 @@ const configPath = path.join(webRoot, '/themes/uppy/_config.yml')
 const { version } = require(path.join(uppyRoot, '/package.json'))
 
 const defaultConfig = {
-  comment: 'Auto updated by update.js',
+  comment: 'Auto updated by inject.js',
   uppy_version_anchor: '001',
   uppy_version: '0.0.1',
   uppy_bundle_kb_sizes: {},
@@ -57,7 +57,7 @@ const excludes = {
   '@uppy/react': ['react']
 }
 
-update().catch((err) => {
+inject().catch((err) => {
   console.error(err)
   process.exit(1)
 })
@@ -81,7 +81,7 @@ async function getMinifiedSize (pkg, name) {
   }
 }
 
-async function updateSizes (config) {
+async function injectSizes (config) {
   console.info(chalk.grey('Generating bundle sizes…'))
   const padTarget = packages.reduce((max, cur) => Math.max(max, cur.length), 0) + 2
 
@@ -110,7 +110,7 @@ async function updateSizes (config) {
   config.uppy_bundle_kb_sizes = await sizesPromise
 }
 
-async function injectBuiltFiles () {
+async function injectBundles () {
   const cmds = [
     `mkdir -p ${path.join(webRoot, '/themes/uppy/source/uppy')}`,
     `cp -vfR ${path.join(uppyRoot, '/dist/*')} ${path.join(webRoot, '/themes/uppy/source/uppy/')}`,
@@ -156,21 +156,21 @@ async function readConfig () {
   }
 }
 
-async function update () {
+async function inject () {
   const config = await readConfig()
 
   await injectMarkdown()
 
   config.uppy_version = version
   config.uppy_version_anchor = version.replace(/[^\d]+/g, '')
-  await updateSizes(config)
+  await injectSizes(config)
 
   const saveConfig = Object.assign({}, defaultConfig, config)
   await promisify(fs.writeFile)(configPath, YAML.safeDump(saveConfig), 'utf-8')
   console.info(chalk.green('✓ rewritten: '), chalk.grey(configPath))
 
   try {
-    await injectBuiltFiles()
+    await injectBundles()
   } catch (error) {
     console.error(
       chalk.red('x failed to inject: '),

website/src/_template/contributing.md

@@ -41,13 +41,13 @@ selenium-standalone start
 After you have installed and launched the selenium standalone server, run:
 
 ```bash
-npm run test:acceptance:local
+npm run test:endtoend:local
 ```
 
-By default, `test:acceptance:local` uses Firefox. You can use a different browser, like Chrome, by passing the `-b` flag:
+By default, `test:endtoend:local` uses Firefox. You can use a different browser, like Chrome, by passing the `-b` flag:
 
 ```bash
-npm run test:acceptance:local -- -b chrome
+npm run test:endtoend:local -- -b chrome
 ```
 
 > Note: The `--` is important, it tells npm that the remaining arguments should be interpreted by the script itself, not by npm.
@@ -55,13 +55,13 @@ npm run test:acceptance:local -- -b chrome
 You can run in multiple browsers by passing multiple `-b` flags:
 
 ```bash
-npm run test:acceptance:local -- -b chrome -b firefox
+npm run test:endtoend:local -- -b chrome -b firefox
 ```
 
 When trying to get a specific integration test to pass, it's not that helpful to continuously run _all_ tests. You can use the `--suite` flag to run tests from a single `./test/endtoend` folder. For example, `--suite thumbnails` will only run the tests from `./test/endtoend/thumbnails`. Of course, it can also be combined with one or more `-b` flags.
 
 ```bash
-npm run test:acceptance:local -- -b chrome --suite thumbnails
+npm run test:endtoend:local -- -b chrome --suite thumbnails
 ```
 
 These tests are also run automatically on Travis builds with [SauceLabs](https://saucelabs.com/) cloud service using different OSes.

website/src/api-usage-example.js

@@ -1,18 +0,0 @@
-import Uppy from '@uppy/core'
-import Dashboard from '@uppy/dashboard'
-import Instagram from '@uppy/instagram'
-import Tus from '@uppy/tus'
-
-Uppy()
-  .use(Dashboard, {
-    trigger: '#select-files',
-    showProgressDetails: true
-  })
-  .use(Instagram, {
-    target: Dashboard,
-    serverUrl: 'https://companion.uppy.io'
-  })
-  .use(Tus, { endpoint: 'https://master.tus.io/files/' })
-  .on('complete', (result) => {
-    console.log('Upload result:', result)
-  })

website/src/frontpage-code-sample.ejs

@@ -2,11 +2,30 @@
 layout: false
 ---
 <!--
-You can `npm run web:inject:frontpagecodesample` to render this code snippet and
+You can `npm run web:inject-frontpagecodesample` to render this code snippet and
 save it as a layout partial
 -->
 {% codeblock lang:bash %}
 $ npm install @uppy/core @uppy/dashboard @uppy/instagram @uppy/tus 
 {% endcodeblock %}
 
-{% include_code lang:js ../api-usage-example.js %}
+{% codeblock lang:js %}
+import Uppy from '@uppy/core'
+import Dashboard from '@uppy/dashboard'
+import Instagram from '@uppy/instagram'
+import Tus from '@uppy/tus'
+
+Uppy()
+  .use(Dashboard, {
+    trigger: '#select-files',
+    showProgressDetails: true
+  })
+  .use(Instagram, {
+    target: Dashboard,
+    serverUrl: 'https://companion.uppy.io'
+  })
+  .use(Tus, { endpoint: 'https://master.tus.io/files/' })
+  .on('complete', (result) => {
+    console.log('Upload result:', result)
+  })
+{% endcodeblock %}

website/themes/uppy/layout/partials/frontpage-code-sample.html

@@ -1,5 +1,5 @@
 <!--
-You can `npm run web:inject:frontpagecodesample` to render this code snippet and
+You can `npm run web:inject-frontpagecodesample` to render this code snippet and
 save it as a layout partial
 -->
 <figure class="highlight bash"><table><tr><td class="code"><pre>$ <span class="token function">npm</span> <span class="token function">install</span> @uppy/core @uppy/dashboard @uppy/instagram @uppy/tus </pre></td></tr></table></figure>