FOSS
/
urltomarkdown
mirror of https://github.com/macsplit/urltomarkdown
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

added rate limiting

main
Lee Hanken 2022-01-09 15:20:59 +00:00
parent 87f476f29b
commit 3b2a262a72
12 changed files with 1558 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
index.js
View File

@ -5,10 +5,21 @@ const JSDOM = require('jsdom').JSDOM;
service = new turndown();
const rateLimit = require('express-rate-limit');
const rateLimiter = rateLimit({
windowMs: 30 * 1000,
max: 5,
message: 'Rate limit exceeded',
headers: true
});
const express = require('express')
const app = express()
const port = process.env.PORT
app.use(rateLimiter)
app.get('/', (req, res) => {
url = req.query.url;
res.header("Access-Control-Allow-Origin", '*');

11
node_modules/.package-lock.json generated vendored
View File

@ -406,6 +406,17 @@
"node": ">= 0.10.0"
}
},
"node_modules/express-rate-limit": {
"version": "6.0.5",
"resolved": "https://registry.npmjs.org/express-rate-limit/-/express-rate-limit-6.0.5.tgz",
"integrity": "sha512-EB1mRTrzyyPfEsQZIQFXocd8NKZoDZbEwrtbdgkc20Yed6oYg02Xfjza2HHPI/0orp54BrFeHeT92ICB9ydokw==",
"engines": {
"node": ">= 14.5.0"
},
"peerDependencies": {
"express": "^4"
}
},
"node_modules/fast-levenshtein": {
"version": "2.0.6",
"resolved": "https://registry.npmjs.org/fast-levenshtein/-/fast-levenshtein-2.0.6.tgz",

153
node_modules/express-rate-limit/changelog.md generated vendored Normal file
View File

@ -0,0 +1,153 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [6.0.5](https://github.com/nfriedly/express-rate-limit/releases/tag/v6.0.5)
### Fixed
- Use named imports for ExpressJS types so users do not need to enable the
`esModuleInterop` flag in their Typescript compiler configuration.
## [6.0.4](https://github.com/nfriedly/express-rate-limit/releases/tag/v6.0.4)
### Fixed
- Upload the built package as a `.tgz` to GitHub releases.
### Changed
- Add ` main` and `module` fields to `package.json`. This helps tools such as
ESLint that do not yet support the `exports` field.
- Bumped the minimum node.js version in `package-lock.json` to match
`package.json`
## [6.0.3](https://github.com/nfriedly/express-rate-limit/releases/tag/v6.0.3)
### Changed
- Bumped minimum Node version from 12.9 to 14.5 in `package.json` because the
transpiled output uses the nullish coalescing operator (`??`), which
[isn't supported in node.js prior to 14.x](https://node.green/#ES2020-features--nullish-coalescing-operator-----).
## [6.0.2](https://github.com/nfriedly/express-rate-limit/releases/v6.0.2)
### Fixed
- Ensure CommonJS projects can import the module.
### Added
- Add additional tests that test:
- importing the library in `js-cjs`, `js-esm`, `ts-cjs`, `ts-esm`
environments.
- usage of the library with external stores (`redis`, `mongo`, `memcached`,
`precise`).
### Changed
- Use [`esbuild`](https://esbuild.github.io/) to generate ESM and CJS output.
This reduces the size of the built package from 138 kb to 13kb and build time
to 4 ms! :rocket:
- Use [`dts-bundle-generator`](https://github.com/timocov/dts-bundle-generator)
to generate a single Typescript declaration file.
## [6.0.1](https://github.com/nfriedly/express-rate-limit/releases/v6.0.1)
### Fixed
- Ensure CommonJS projects can import the module.
## [6.0.0](https://github.com/nfriedly/express-rate-limit/releases/v6.0.0)
### Added
- `express` 4.x as a peer dependency.
- Better Typescript support (the library was rewritten in Typescript).
- Export the package as both ESM and CJS.
- Publish the built package (`.tgz` file) on GitHub releases as well as the npm
registry.
- Issue and PR templates.
- A contributing guide.
### Changed
- Rename the `draft_polli_ratelimit_headers` option to `standardHeaders`.
- Rename the `headers` option to `legacyHeaders`.
- `Retry-After` header is now sent if either `legacyHeaders` or
`standardHeaders` is set.
- Allow `keyGenerator` to be an async function/return a promise.
- Change the way custom stores are defined.
- Add the `init` method for stores to set themselves up using options passed
to the middleware.
- Rename the `incr` method to `increment`.
- Allow the `increment`, `decrement`, `resetKey` and `resetAll` methods to
return a promise.
- Old stores will automatically be promisified and used.
- The package can now only be used with NodeJS version 12.9.0 or greater.
- The `onLimitReached` configuration option is now deprecated. Replace it with a
custom `handler` that checks the number of hits.
### Removed
- Remove the deprecated `limiter.resetIp` method (use the `limiter.resetKey`
method instead).
- Remove the deprecated options `delayMs`, `delayAfter` (the delay functionality
was moved to the
[`express-slow-down`](https://github.com/nfriedly/express-slow-down) package)
and `global` (use a key generator that returns a constant value).
## [5.x](https://github.com/nfriedly/express-rate-limit/releases/tag/v5.5.1)
### Added
- The middleware ~throws~ logs an error if `request.ip` is undefined.
### Removed
- Removes typescript typings. (See
[#138](https://github.com/nfriedly/express-rate-limit/issues/138))
## [4.x](https://github.com/nfriedly/express-rate-limit/releases/tag/v4.0.4)
### Changed
- The library no longer modifies the passed-in options object, it instead makes
a clone of it.
## [3.x](https://github.com/nfriedly/express-rate-limit/releases/tag/v3.5.2)
### Added
- Simplifies the default `handler` function so that it no longer changes the
response format. The default handler also uses
[response.send](https://expressjs.com/en/4x/api.html#response.send).
### Changes
- `onLimitReached` now only triggers once for a client and window. However, the
`handle` method is called for every blocked request.
### Removed
- The `delayAfter` and `delayMs` options; they were moved to the
[express-slow-down](https://npmjs.org/package/express-slow-down) package.
## [2.x](https://github.com/nfriedly/express-rate-limit/releases/tag/v2.14.2)
### Added
- A `limiter.resetKey()` method to reset the hit counter for a particular client
### Changes
- The rate limiter now uses a less precise but less resource intensive method of
tracking hits from a client.
### Removed
- The `global` option.

226
node_modules/express-rate-limit/dist/index.cjs generated vendored Normal file
View File

@ -0,0 +1,226 @@
var __defProp = Object.defineProperty;
var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
var __getOwnPropNames = Object.getOwnPropertyNames;
var __hasOwnProp = Object.prototype.hasOwnProperty;
var __markAsModule = (target) => __defProp(target, "__esModule", { value: true });
var __export = (target, all) => {
for (var name in all)
__defProp(target, name, { get: all[name], enumerable: true });
};
var __reExport = (target, module2, copyDefault, desc) => {
if (module2 && typeof module2 === "object" || typeof module2 === "function") {
for (let key of __getOwnPropNames(module2))
if (!__hasOwnProp.call(target, key) && (copyDefault || key !== "default"))
__defProp(target, key, { get: () => module2[key], enumerable: !(desc = __getOwnPropDesc(module2, key)) || desc.enumerable });
}
return target;
};
var __toCommonJS = /* @__PURE__ */ ((cache) => {
return (module2, temp) => {
return cache && cache.get(module2) || (temp = __reExport(__markAsModule({}), module2, 1), cache && cache.set(module2, temp), temp);
};
})(typeof WeakMap !== "undefined" ? /* @__PURE__ */ new WeakMap() : 0);
// source/index.ts
var source_exports = {};
__export(source_exports, {
default: () => source_default
});
// source/memory-store.ts
var calculateNextResetTime = (windowMs) => {
const resetTime = new Date();
resetTime.setMilliseconds(resetTime.getMilliseconds() + windowMs);
return resetTime;
};
var MemoryStore = class {
init(options) {
this.windowMs = options.windowMs;
this.resetTime = calculateNextResetTime(this.windowMs);
this.hits = {};
const interval = setInterval(async () => {
await this.resetAll();
}, this.windowMs);
if (interval.unref) {
interval.unref();
}
}
async increment(key) {
const totalHits = (this.hits[key] ?? 0) + 1;
this.hits[key] = totalHits;
return {
totalHits,
resetTime: this.resetTime
};
}
async decrement(key) {
const current = this.hits[key];
if (current) {
this.hits[key] = current - 1;
}
}
async resetKey(key) {
delete this.hits[key];
}
async resetAll() {
this.hits = {};
this.resetTime = calculateNextResetTime(this.windowMs);
}
};
// source/lib.ts
var isLegacyStore = (store) => typeof store.incr === "function" && typeof store.increment !== "function";
var promisifyStore = (passedStore) => {
if (!isLegacyStore(passedStore)) {
return passedStore;
}
const legacyStore = passedStore;
class PromisifiedStore {
async increment(key) {
return new Promise((resolve, reject) => {
legacyStore.incr(key, (error, totalHits, resetTime) => {
if (error)
reject(error);
resolve({ totalHits, resetTime });
});
});
}
async decrement(key) {
return Promise.resolve(legacyStore.decrement(key));
}
async resetKey(key) {
return Promise.resolve(legacyStore.resetKey(key));
}
async resetAll() {
if (typeof legacyStore.resetAll === "function")
return Promise.resolve(legacyStore.resetAll());
}
}
return new PromisifiedStore();
};
var parseOptions = (passedOptions) => {
const options = {
windowMs: 60 * 1e3,
store: new MemoryStore(),
max: 5,
message: "Too many requests, please try again later.",
statusCode: 429,
legacyHeaders: passedOptions.headers ?? true,
standardHeaders: passedOptions.draft_polli_ratelimit_headers ?? false,
requestPropertyName: "rateLimit",
skipFailedRequests: false,
skipSuccessfulRequests: false,
requestWasSuccessful: (_request, response) => response.statusCode < 400,
skip: (_request, _response) => false,
keyGenerator: (request, _response) => {
if (!request.ip) {
console.error("WARN | `express-rate-limit` | `request.ip` is undefined. You can avoid this by providing a custom `keyGenerator` function, but it may be indicative of a larger issue.");
}
return request.ip;
},
handler: (_request, response, _next, _optionsUsed) => {
response.status(options.statusCode).send(options.message);
},
onLimitReached: (_request, _response, _optionsUsed) => {
},
...passedOptions
};
if (typeof options.store.incr !== "function" && typeof options.store.increment !== "function" || typeof options.store.decrement !== "function" || typeof options.store.resetKey !== "function" || typeof options.store.resetAll !== "undefined" && typeof options.store.resetAll !== "function" || typeof options.store.init !== "undefined" && typeof options.store.init !== "function") {
throw new TypeError("An invalid store was passed. Please ensure that the store is a class that implements the `Store` interface.");
}
options.store = promisifyStore(options.store);
return options;
};
var handleAsyncErrors = (fn) => async (request, response, next) => {
try {
await Promise.resolve(fn(request, response, next)).catch(next);
} catch (error) {
next(error);
}
};
var rateLimit = (passedOptions) => {
const options = parseOptions(passedOptions ?? {});
if (typeof options.store.init === "function")
options.store.init(options);
const middleware = handleAsyncErrors(async (request, response, next) => {
const skip = await options.skip(request, response);
if (skip) {
next();
return;
}
const augmentedRequest = request;
const key = await options.keyGenerator(request, response);
const { totalHits, resetTime } = await options.store.increment(key);
const retrieveQuota = typeof options.max === "function" ? options.max(request, response) : options.max;
const maxHits = await retrieveQuota;
augmentedRequest[options.requestPropertyName] = {
limit: maxHits,
current: totalHits,
remaining: Math.max(maxHits - totalHits, 0),
resetTime
};
if (options.legacyHeaders && !response.headersSent) {
response.setHeader("X-RateLimit-Limit", maxHits);
response.setHeader("X-RateLimit-Remaining", augmentedRequest[options.requestPropertyName].remaining);
if (resetTime instanceof Date) {
response.setHeader("Date", new Date().toUTCString());
response.setHeader("X-RateLimit-Reset", Math.ceil(resetTime.getTime() / 1e3));
}
}
if (options.standardHeaders && !response.headersSent) {
response.setHeader("RateLimit-Limit", maxHits);
response.setHeader("RateLimit-Remaining", augmentedRequest[options.requestPropertyName].remaining);
if (resetTime) {
const deltaSeconds = Math.ceil((resetTime.getTime() - Date.now()) / 1e3);
response.setHeader("RateLimit-Reset", Math.max(0, deltaSeconds));
}
}
if (options.skipFailedRequests || options.skipSuccessfulRequests) {
let decremented = false;
const decrementKey = async () => {
if (!decremented) {
await options.store.decrement(key);
decremented = true;
}
};
if (options.skipFailedRequests) {
response.on("finish", async () => {
if (!options.requestWasSuccessful(request, response))
await decrementKey();
});
response.on("close", async () => {
if (!response.writableEnded)
await decrementKey();
});
response.on("error", async () => {
await decrementKey();
});
}
if (options.skipSuccessfulRequests) {
response.on("finish", async () => {
if (options.requestWasSuccessful(request, response))
await decrementKey();
});
}
}
if (maxHits && totalHits === maxHits + 1) {
options.onLimitReached(request, response, options);
}
if (maxHits && totalHits > maxHits) {
if ((options.legacyHeaders || options.standardHeaders) && !response.headersSent) {
response.setHeader("Retry-After", Math.ceil(options.windowMs / 1e3));
}
options.handler(request, response, next, options);
return;
}
next();
});
middleware.resetKey = options.store.resetKey.bind(options.store);
return middleware;
};
var lib_default = rateLimit;
// source/index.ts
var source_default = lib_default;
module.exports = __toCommonJS(source_exports);
module.exports = rateLimit;

248
node_modules/express-rate-limit/dist/index.d.ts generated vendored Normal file
View File

@ -0,0 +1,248 @@
// Generated by dts-bundle-generator v6.3.0
import { NextFunction, Request, RequestHandler, Response } from 'express';
/**
* Callback that fires when a client's hit counter is incremented.
*
* @param error {Error | undefined} - The error that occurred, if any
* @param totalHits {number} - The number of hits for that client so far
* @param resetTime {Date | undefined} - The time when the counter resets
*/
export declare type IncrementCallback = (error: Error | undefined, totalHits: number, resetTime: Date | undefined) => void;
/**
* Method (in the form of middleware) to generate/retrieve a value based on the
* incoming request
*
* @param request {Request} - The Express request object
* @param response {Response} - The Express response object
*
* @returns {T} - The value needed
*/
export declare type ValueDeterminingMiddleware<T> = (request: Request, response: Response) => T | Promise<T>;
/**
* Express request handler that sends back a response when a client is
* rate-limited.
*
* @param request {Request} - The Express request object
* @param response {Response} - The Express response object
* @param next {NextFunction} - The Express `next` function, can be called to skip responding
* @param optionsUsed {Options} - The options used to set up the middleware
*/
export declare type RateLimitExceededEventHandler = (request: Request, response: Response, next: NextFunction, optionsUsed: Options) => void;
/**
* Event callback that is triggered on a client's first request that exceeds the limit
* but not for subsequent requests. May be used for logging, etc. Should *not*
* send a response.
*
* @param request {Request} - The Express request object
* @param response {Response} - The Express response object
* @param optionsUsed {Options} - The options used to set up the middleware
*/
export declare type RateLimitReachedEventHandler = (request: Request, response: Response, optionsUsed: Options) => void;
/**
* Data returned from the `Store` when a client's hit counter is incremented.
*
* @property totalHits {number} - The number of hits for that client so far
* @property resetTime {Date | undefined} - The time when the counter resets
*/
export declare type IncrementResponse = {
totalHits: number;
resetTime: Date | undefined;
};
/**
* A modified Express request handler with the rate limit functions.
*/
export declare type RateLimitRequestHandler = RequestHandler & {
/**
* Method to reset a client's hit counter.
*
* @param key {string} - The identifier for a client
*/
resetKey: (key: string) => void;
};
/**
* An interface that all hit counter stores must implement.
*
* @deprecated 6.x - Implement the `Store` interface instead.
*/
export interface LegacyStore {
/**
* Method to increment a client's hit counter.
*
* @param key {string} - The identifier for a client
* @param callback {IncrementCallback} - The callback to call once the counter is incremented
*/
incr: (key: string, callback: IncrementCallback) => void;
/**
* Method to decrement a client's hit counter.
*
* @param key {string} - The identifier for a client
*/
decrement: (key: string) => void;
/**
* Method to reset a client's hit counter.
*
* @param key {string} - The identifier for a client
*/
resetKey: (key: string) => void;
/**
* Method to reset everyone's hit counter.
*/
resetAll?: () => void;
}
/**
* An interface that all hit counter stores must implement.
*/
export interface Store {
/**
* Method that initializes the store, and has access to the options passed to
* the middleware too.
*
* @param options {Options} - The options used to setup the middleware
*/
init?: (options: Options) => void;
/**
* Method to increment a client's hit counter.
*
* @param key {string} - The identifier for a client
*
* @returns {IncrementResponse} - The number of hits and reset time for that client
*/
increment: (key: string) => Promise<IncrementResponse> | IncrementResponse;
/**
* Method to decrement a client's hit counter.
*
* @param key {string} - The identifier for a client
*/
decrement: (key: string) => Promise<void> | void;
/**
* Method to reset a client's hit counter.
*
* @param key {string} - The identifier for a client
*/
resetKey: (key: string) => Promise<void> | void;
/**
* Method to reset everyone's hit counter.
*/
resetAll?: () => Promise<void> | void;
}
/**
* The configuration options for the rate limiter.
*/
export interface Options {
/**
* How long we should remember the requests.
*/
readonly windowMs: number;
/**
* The maximum number of connection to allow during the `window` before
* rate limiting the client.
*
* Can be the limit itself as a number or express middleware that parses
* the request and then figures out the limit.
*/
readonly max: number | ValueDeterminingMiddleware<number>;
/**
* The response body to send back when a client is rate limited.
*/
readonly message: any;
/**
* The HTTP status code to send back when a client is rate limited.
*
* Defaults to `HTTP 429 Too Many Requests` (RFC 6585).
*/
readonly statusCode: number;
/**
* Whether to send `X-RateLimit-*` headers with the rate limit and the number
* of requests.
*/
readonly legacyHeaders: boolean;
/**
* Whether to enable support for the rate limit standardization headers (`RateLimit-*`).
*/
readonly standardHeaders: boolean;
/**
* The name of the property on the request object to store the rate limit info.
*
* Defaults to `rateLimit`.
*/
readonly requestPropertyName: string;
/**
* If `true`, the library will (by default) skip all requests that have a 4XX
* or 5XX status.
*/
readonly skipFailedRequests: boolean;
/**
* If `true`, the library will (by default) skip all requests that have a
* status code less than 400.
*/
readonly skipSuccessfulRequests: boolean;
/**
* Method to determine whether or not the request counts as 'succesful'. Used
* when either `skipSuccessfulRequests` or `skipFailedRequests` is set to true.
*/
readonly requestWasSuccessful: ValueDeterminingMiddleware<boolean>;
/**
* Method to generate custom identifiers for clients.
*
* By default, the client's IP address is used.
*/
readonly keyGenerator: ValueDeterminingMiddleware<string>;
/**
* Method (in the form of middleware) to determine whether or not this request
* counts towards a client's quota.
*/
readonly skip: ValueDeterminingMiddleware<boolean>;
/**
* Express request handler that sends back a response when a client is
* rate-limited.
*/
readonly handler: RateLimitExceededEventHandler;
/**
* Express request handler that sends back a response when a client has
* reached their rate limit, and will be rate limited on their next request.
*/
readonly onLimitReached: RateLimitReachedEventHandler;
/**
* The {@link Store} to use to store the hit count for each client.
*/
store: Store;
/**
* Whether to send `X-RateLimit-*` headers with the rate limit and the number
* of requests.
*
* @deprecated 6.x - This option was renamed to `legacyHeaders`.
*/
headers?: boolean;
/**
* Whether to send `RateLimit-*` headers with the rate limit and the number
* of requests.
*
* @deprecated 6.x - This option was renamed to `standardHeaders`.
*/
draft_polli_ratelimit_headers?: boolean;
}
/**
* The extended request object that includes information about the client's
* rate limit.
*/
export declare type AugmentedRequest = Request & {
[key: string]: RateLimitInfo;
};
/**
* The rate limit related information for each client included in the
* Express request object.
*/
export interface RateLimitInfo {
readonly limit: number;
readonly current: number;
readonly remaining: number;
readonly resetTime: Date | undefined;
}
declare const rateLimit: (passedOptions?: (Omit<Partial<Options>, "store"> & {
store?: LegacyStore | Store | undefined;
}) | undefined) => RateLimitRequestHandler;
export default rateLimit;
export {};

198
node_modules/express-rate-limit/dist/index.mjs generated vendored Normal file
View File

@ -0,0 +1,198 @@
// source/memory-store.ts
var calculateNextResetTime = (windowMs) => {
const resetTime = new Date();
resetTime.setMilliseconds(resetTime.getMilliseconds() + windowMs);
return resetTime;
};
var MemoryStore = class {
init(options) {
this.windowMs = options.windowMs;
this.resetTime = calculateNextResetTime(this.windowMs);
this.hits = {};
const interval = setInterval(async () => {
await this.resetAll();
}, this.windowMs);
if (interval.unref) {
interval.unref();
}
}
async increment(key) {
const totalHits = (this.hits[key] ?? 0) + 1;
this.hits[key] = totalHits;
return {
totalHits,
resetTime: this.resetTime
};
}
async decrement(key) {
const current = this.hits[key];
if (current) {
this.hits[key] = current - 1;
}
}
async resetKey(key) {
delete this.hits[key];
}
async resetAll() {
this.hits = {};
this.resetTime = calculateNextResetTime(this.windowMs);
}
};
// source/lib.ts
var isLegacyStore = (store) => typeof store.incr === "function" && typeof store.increment !== "function";
var promisifyStore = (passedStore) => {
if (!isLegacyStore(passedStore)) {
return passedStore;
}
const legacyStore = passedStore;
class PromisifiedStore {
async increment(key) {
return new Promise((resolve, reject) => {
legacyStore.incr(key, (error, totalHits, resetTime) => {
if (error)
reject(error);
resolve({ totalHits, resetTime });
});
});
}
async decrement(key) {
return Promise.resolve(legacyStore.decrement(key));
}
async resetKey(key) {
return Promise.resolve(legacyStore.resetKey(key));
}
async resetAll() {
if (typeof legacyStore.resetAll === "function")
return Promise.resolve(legacyStore.resetAll());
}
}
return new PromisifiedStore();
};
var parseOptions = (passedOptions) => {
const options = {
windowMs: 60 * 1e3,
store: new MemoryStore(),
max: 5,
message: "Too many requests, please try again later.",
statusCode: 429,
legacyHeaders: passedOptions.headers ?? true,
standardHeaders: passedOptions.draft_polli_ratelimit_headers ?? false,
requestPropertyName: "rateLimit",
skipFailedRequests: false,
skipSuccessfulRequests: false,
requestWasSuccessful: (_request, response) => response.statusCode < 400,
skip: (_request, _response) => false,
keyGenerator: (request, _response) => {
if (!request.ip) {
console.error("WARN | `express-rate-limit` | `request.ip` is undefined. You can avoid this by providing a custom `keyGenerator` function, but it may be indicative of a larger issue.");
}
return request.ip;
},
handler: (_request, response, _next, _optionsUsed) => {
response.status(options.statusCode).send(options.message);
},
onLimitReached: (_request, _response, _optionsUsed) => {
},
...passedOptions
};
if (typeof options.store.incr !== "function" && typeof options.store.increment !== "function" || typeof options.store.decrement !== "function" || typeof options.store.resetKey !== "function" || typeof options.store.resetAll !== "undefined" && typeof options.store.resetAll !== "function" || typeof options.store.init !== "undefined" && typeof options.store.init !== "function") {
throw new TypeError("An invalid store was passed. Please ensure that the store is a class that implements the `Store` interface.");
}
options.store = promisifyStore(options.store);
return options;
};
var handleAsyncErrors = (fn) => async (request, response, next) => {
try {
await Promise.resolve(fn(request, response, next)).catch(next);
} catch (error) {
next(error);
}
};
var rateLimit = (passedOptions) => {
const options = parseOptions(passedOptions ?? {});
if (typeof options.store.init === "function")
options.store.init(options);
const middleware = handleAsyncErrors(async (request, response, next) => {
const skip = await options.skip(request, response);
if (skip) {
next();
return;
}
const augmentedRequest = request;
const key = await options.keyGenerator(request, response);
const { totalHits, resetTime } = await options.store.increment(key);
const retrieveQuota = typeof options.max === "function" ? options.max(request, response) : options.max;
const maxHits = await retrieveQuota;
augmentedRequest[options.requestPropertyName] = {
limit: maxHits,
current: totalHits,
remaining: Math.max(maxHits - totalHits, 0),
resetTime
};
if (options.legacyHeaders && !response.headersSent) {
response.setHeader("X-RateLimit-Limit", maxHits);
response.setHeader("X-RateLimit-Remaining", augmentedRequest[options.requestPropertyName].remaining);
if (resetTime instanceof Date) {
response.setHeader("Date", new Date().toUTCString());
response.setHeader("X-RateLimit-Reset", Math.ceil(resetTime.getTime() / 1e3));
}
}
if (options.standardHeaders && !response.headersSent) {
response.setHeader("RateLimit-Limit", maxHits);
response.setHeader("RateLimit-Remaining", augmentedRequest[options.requestPropertyName].remaining);
if (resetTime) {
const deltaSeconds = Math.ceil((resetTime.getTime() - Date.now()) / 1e3);
response.setHeader("RateLimit-Reset", Math.max(0, deltaSeconds));
}
}
if (options.skipFailedRequests || options.skipSuccessfulRequests) {
let decremented = false;
const decrementKey = async () => {
if (!decremented) {
await options.store.decrement(key);
decremented = true;
}
};
if (options.skipFailedRequests) {
response.on("finish", async () => {
if (!options.requestWasSuccessful(request, response))
await decrementKey();
});
response.on("close", async () => {
if (!response.writableEnded)
await decrementKey();
});
response.on("error", async () => {
await decrementKey();
});
}
if (options.skipSuccessfulRequests) {
response.on("finish", async () => {
if (options.requestWasSuccessful(request, response))
await decrementKey();
});
}
}
if (maxHits && totalHits === maxHits + 1) {
options.onLimitReached(request, response, options);
}
if (maxHits && totalHits > maxHits) {
if ((options.legacyHeaders || options.standardHeaders) && !response.headersSent) {
response.setHeader("Retry-After", Math.ceil(options.windowMs / 1e3));
}
options.handler(request, response, next, options);
return;
}
next();
});
middleware.resetKey = options.store.resetKey.bind(options.store);
return middleware;
};
var lib_default = rateLimit;
// source/index.ts
var source_default = lib_default;
export {
source_default as default
};

20
node_modules/express-rate-limit/license.md generated vendored Normal file
View File

@ -0,0 +1,20 @@
# MIT License
Copyright 2021 Nathan Friedly
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

144
node_modules/express-rate-limit/package.json generated vendored Normal file
View File

@ -0,0 +1,144 @@
{
"name": "express-rate-limit",
"version": "6.0.5",
"description": "Basic IP rate-limiting middleware for Express. Use to limit repeated requests to public APIs and/or endpoints such as password reset.",
"author": {
"name": "Nathan Friedly",
"url": "http://nfriedly.com/"
},
"license": "MIT",
"homepage": "https://github.com/nfriedly/express-rate-limit",
"repository": "https://github.com/nfriedly/express-rate-limit",
"keywords": [
"express-rate-limit",
"express",
"rate",
"limit",
"ratelimit",
"rate-limit",
"middleware",
"ip",
"auth",
"authorization",
"security",
"brute",
"force",
"bruteforce",
"brute-force",
"attack"
],
"type": "module",
"exports": {
".": {
"import": "./dist/index.mjs",
"require": "./dist/index.cjs"
}
},
"main": "./dist/index.cjs",
"module": "./dist/index.mjs",
"types": "./dist/index.d.ts",
"files": [
"dist/",
"tsconfig.json",
"package.json",
"readme.md",
"license.md",
"changelog.md"
],
"engines": {
"node": ">= 14.5.0"
},
"scripts": {
"clean": "del-cli dist/ coverage/ *.log *.tmp *.bak *.tgz",
"build:cjs": "esbuild --bundle --format=cjs --outfile=dist/index.cjs --footer:js=\"module.exports = rateLimit;\" source/index.ts",
"build:esm": "esbuild --bundle --format=esm --outfile=dist/index.mjs source/index.ts",
"build:types": "dts-bundle-generator --out-file=dist/index.d.ts source/index.ts",
"compile": "run-s clean build:*",
"lint:code": "xo --ignore test/external/",
"lint:rest": "prettier --ignore-path .gitignore --ignore-unknown --check .",
"lint": "run-s lint:*",
"autofix:code": "xo --ignore test/external/ --fix",
"autofix:rest": "prettier --ignore-path .gitignore --ignore-unknown --write .",
"autofix": "run-s autofix:*",
"test:lib": "cross-env NODE_OPTIONS=--experimental-vm-modules jest",
"test:ext": "npm pack && cd test/external/ && bash run-all-tests",
"test": "run-s lint test:*",
"pre-commit": "lint-staged",
"prepare": "run-s compile && husky install config/husky"
},
"peerDependencies": {
"express": "^4"
},
"devDependencies": {
"@jest/globals": "^27.4.6",
"@types/express": "^4.17.13",
"@types/jest": "^27.4.0",
"@types/node": "^16.11.17",
"@types/supertest": "^2.0.11",
"cross-env": "^7.0.3",
"del-cli": "^4.0.1",
"dts-bundle-generator": "^6.3.0",
"esbuild": "^0.14.10",
"express": "^4.17.1",
"husky": "^7.0.4",
"jest": "^27.4.7",
"lint-staged": "^12.1.5",
"npm-run-all": "^4.1.5",
"supertest": "^6.1.6",
"ts-jest": "^27.1.1",
"ts-node": "^10.4.0",
"typescript": "^4.5.2",
"xo": "^0.47.0"
},
"xo": {
"prettier": true,
"rules": {
"@typescript-eslint/no-empty-function": 0,
"@typescript-eslint/no-dynamic-delete": 0,
"@typescript-eslint/no-confusing-void-expression": 0,
"@typescript-eslint/consistent-indexed-object-style": [
"error",
"index-signature"
]
}
},
"prettier": {
"semi": false,
"useTabs": true,
"singleQuote": true,
"bracketSpacing": true,
"trailingComma": "all",
"proseWrap": "always"
},
"jest": {
"preset": "ts-jest/presets/default-esm",
"globals": {
"ts-jest": {
"useESM": true
}
},
"verbose": true,
"collectCoverage": true,
"collectCoverageFrom": [
"source/**/*.ts"
],
"testTimeout": 30000,
"testMatch": [
"**/test/library/**/*-test.[jt]s?(x)"
],
"moduleFileExtensions": [
"js",
"jsx",
"json",
"ts",
"tsx"
],
"moduleNameMapper": {
"^(\\.{1,2}/.*)\\.js$": "$1"
}
},
"lint-staged": {
"{source,test}/**/*.ts": "xo --ignore test/external/ --fix",
"**/*.{json,yaml,md}": "prettier --ignore-path .gitignore --ignore-unknown --write "
}
}

515
node_modules/express-rate-limit/readme.md generated vendored Normal file
View File

@ -0,0 +1,515 @@
# <div align="center"> Express Rate Limit </div>
<div align="center">
[![Tests](https://github.com/nfriedly/express-rate-limit/workflows/Test/badge.svg)](https://github.com/nfriedly/express-rate-limit/actions)
[![npm version](https://img.shields.io/npm/v/express-rate-limit.svg)](https://npmjs.org/package/express-rate-limit 'View this project on NPM')
[![npm downloads](https://img.shields.io/npm/dm/express-rate-limit)](https://www.npmjs.com/package/express-rate-limit)
Basic rate-limiting middleware for Express. Use to limit repeated requests to
public APIs and/or endpoints such as password reset. Plays nice with
[express-slow-down](https://www.npmjs.com/package/express-slow-down).
</div>
### Alternate Rate Limiters
> This module does not share state with other processes/servers by default. If
> you need a more robust solution, I recommend using an external store. See the
> [`stores` section](#store) below for a list of external stores.
This module was designed to only handle the basics and didn't even support
external stores initially. These other options all are excellent pieces of
software and may be more appropriate for some situations:
- [rate-limiter-flexible](https://www.npmjs.com/package/rate-limiter-flexible)
- [express-brute](https://www.npmjs.com/package/express-brute)
- [rate-limiter](https://www.npmjs.com/package/express-limiter)
## Installation
From the npm registry:
```sh
# Using npm
> npm install express-rate-limit
# Using yarn or pnpm
> yarn/pnpm add express-rate-limit
```
From Github Releases:
```sh
# Using npm
> npm install https://github.com/nfriedly/express-rate-limit/releases/download/v{version}/express-rate-limit.tgz
# Using yarn or pnpm
> yarn/pnpm add https://github.com/nfriedly/express-rate-limit/releases/download/v{version}/express-rate-limit.tgz
```
Replace `{version}` with the version of the package that you want to your, e.g.:
`6.0.0`.
## Usage
### Importing
This library is provided in ESM as well as CJS forms, and works with both
Javascript and Typescript projects.
**This package requires you to use Node 14 or above.**
Import it in a CommonJS project (`type: commonjs` or no `type` field in
`package.json`) as follows:
```ts
const rateLimit = require('express-rate-limit')
```
Import it in a ESM project (`type: module` in `package.json`) as follows:
```ts
import rateLimit from 'express-rate-limit'
```
### Examples
To use it in an API-only server where the rate-limiter should be applied to all
requests:
```ts
import rateLimit from 'express-rate-limit'
const limiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // Limit each IP to 100 requests per `window` (here, per 15 minutes)
standardHeaders: true, // Return rate limit info in the `RateLimit-*` headers
legacyHeaders: false, // Disable the `X-RateLimit-*` headers
})
// Apply the rate limiting middleware to all requests
app.use(limiter)
```
To use it in a 'regular' web server (e.g. anything that uses
`express.static()`), where the rate-limiter should only apply to certain
requests:
```ts
import rateLimit from 'express-rate-limit'
const apiLimiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // Limit each IP to 100 requests per `window` (here, per 15 minutes)
standardHeaders: true, // Return rate limit info in the `RateLimit-*` headers
legacyHeaders: false, // Disable the `X-RateLimit-*` headers
})
// Apply the rate limiting middleware to API calls only
app.use('/api', apiLimiter)
```
To create multiple instances to apply different rules to different endpoints:
```ts
import rateLimit from 'express-rate-limit'
const apiLimiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // Limit each IP to 100 requests per `window` (here, per 15 minutes)
standardHeaders: true, // Return rate limit info in the `RateLimit-*` headers
legacyHeaders: false, // Disable the `X-RateLimit-*` headers
})
app.use('/api/', apiLimiter)
const createAccountLimiter = rateLimit({
windowMs: 60 * 60 * 1000, // 1 hour
max: 5, // Limit each IP to 5 create account requests per `window` (here, per hour)
message:
'Too many accounts created from this IP, please try again after an hour',
standardHeaders: true, // Return rate limit info in the `RateLimit-*` headers
legacyHeaders: false, // Disable the `X-RateLimit-*` headers
})
app.post('/create-account', createAccountLimiter, (request, response) => {
//...
})
```
To use a custom store:
```ts
import rateLimit from 'express-rate-limit'
import MemoryStore from 'express-rate-limit/memory-store.js'
const apiLimiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // Limit each IP to 100 requests per `window` (here, per 15 minutes)
standardHeaders: true, // Return rate limit info in the `RateLimit-*` headers
store: new MemoryStore(),
})
// Apply the rate limiting middleware to API calls only
app.use('/api', apiLimiter)
```
> **Note:** most stores will require additional configuration, such as custom
> prefixes, when using multiple instances. The default built-in memory store is
> an exception to this rule.
### Troubleshooting Proxy Issues
If you are behind a proxy/load balancer (usually the case with most hosting
services, e.g. Heroku, Bluemix, AWS ELB, Nginx, Cloudflare, Akamai, Fastly,
Firebase Hosting, Rackspace LB, Riverbed Stingray, etc.), the IP address of the
request might be the IP of the load balancer/reverse proxy (making the rate
limiter effectively a global one and blocking all requests once the limit is
reached) or `undefined`. To solve this issue, add the following line to your
code (right after you create the express application):
```ts
app.set('trust proxy', numberOfProxies)
```
Where `numberOfProxies` is the number of proxies between the user and the
server. To find the correct number, create a test endpoint that returns the
client IP:
```ts
app.set('trust proxy', 1)
app.get('/ip', (request, response) => response.send(request.ip))
```
Go to `/ip` and see the IP address returned in the response. If it matches your
IP address (which you can get by going to http://ip.nfriedly.com/ or
https://api.ipify.org/), then the number of proxies is correct and the rate
limiter should now work correctly. If not, then keep increasing the number until
it does.
For more information about the `trust proxy` setting, take a look at the
[official Express documentation](https://expressjs.com/en/guide/behind-proxies.html).
## Request API
A `request.rateLimit` property is added to all requests with the `limit`,
`current`, and `remaining` number of requests and, if the store provides it, a
`resetTime` Date object. These may be used in your application code to take
additional actions or inform the user of their status.
The property name can be configured with the configuration option
`requestPropertyName`
## Configuration options
### `windowMs`
Time frame for which requests are checked/remembered. Also used in the
`Retry-After` header when the limit is reached.
Note: with non-default stores, you may need to configure this value twice, once
here and once on the store. In some cases the units also differ (e.g. seconds vs
miliseconds)
Defaults to `60000` ms (= 1 minute).
### `max`
Max number of connections during `windowMs` milliseconds before sending a 429
response.
May be a number, or a function that returns a number or a promise. If `max` is a
function, it will be called with `request` and `response` params.
Defaults to `5`. Set to `0` to disable.
Example of using a function:
```ts
import rateLimit from 'express-rate-limit'
const isPremium = (request) => {
// ...
}
const limiter = rateLimit({
// `max` could also be an async function or return a promise
max: (request, response) => {
if (isPremium(request)) return 10
else return 5
},
// ...
})
// Apply the rate limiting middleware to all requests
app.use(limiter)
```
### `message`
Error message sent to user when `max` is exceeded.
May be a `string`, JSON object, or any other value that Express's
[response.send](https://expressjs.com/en/4x/api.html#response.send) method
supports.
Defaults to `'Too many requests, please try again later.'`
### `statusCode`
HTTP status code returned when `max` is exceeded.
Defaults to `429`.
### `legacyHeaders`
Enable headers for request limit (`X-RateLimit-Limit`) and current usage
(`X-RateLimit-Remaining`) on all responses and time to wait before retrying
(`Retry-After`) when `max` is exceeded.
Defaults to `true`.
> Renamed in `6.x` from `headers` to `legacyHeaders`.
### `standardHeaders`
Enable headers conforming to the
[ratelimit standardization draft](https://github.com/ietf-wg-httpapi/ratelimit-headers/blob/main/draft-ietf-httpapi-ratelimit-headers.md)
adopted by the IETF: `RateLimit-Limit`, `RateLimit-Remaining`, and, if the store
supports it, `RateLimit-Reset`. May be used in conjunction with, or instead of
the `legacyHeaders` option.
This setting also enables the `Retry-After` header when `max` is exceeded.
Defaults to `false` (for backward compatibility), but recommended to use.
> Renamed in `6.x` from `draft_polli_ratelimit_headers` to `standardHeaders`.
### `keyGenerator`
Function used to generate keys.
Defaults to `request.ip`, similar to this:
```ts
const keyGenerator = (request /*, response*/) => request.ip
```
### `handler`
The function to handle requests once the max limit is exceeded. It receives the
`request` and the `response` objects. The `next` param is available if you need
to pass to the next middleware/route. Finally, the `options` param has all of
the options that originally passed in when creating the current limiter and the
default values for other options.
The `request.rateLimit` object has `limit`, `current`, and `remaining` number of
requests and, if the store provides it, a `resetTime` Date object.
Defaults to:
```ts
const handler = (request, response, next, options) => {
response.status(options.statusCode).send(options.message)
}
```
### `requestWasSuccessful`
Function that is called when `skipFailedRequests` and/or
`skipSuccessfulRequests` are set to `true`. May be overridden if, for example, a
service sends out a 200 status code on errors.
Defaults to
```ts
const requestWasSuccessful = (request, response) => response.statusCode < 400
```
### `skipFailedRequests`
When set to `true`, failed requests won't be counted. Request considered failed
when:
- response status >= 400
- requests that were cancelled before last chunk of data was sent (response
`close` event triggered)
- response `error` event was triggered by response
(Technically they are counted and then un-counted, so a large number of slow
requests all at once could still trigger a rate-limit. This may be fixed in a
future release.)
Defaults to `false`.
### `skipSuccessfulRequests`
When set to `true` successful requests (response status < 400) won't be counted.
(Technically they are counted and then un-counted, so a large number of slow
requests all at once could still trigger a rate-limit. This may be fixed in a
future release.)
Defaults to `false`.
### `skip`
Function used to skip (whitelist) requests. Returning `true`, or a promise that
resolves with `true`, from the function will skip limiting for that request.
Defaults to always `false` (count all requests):
```ts
const skip = (/*request, response*/) => false
```
### `requestPropertyName`
The name of the property that contains the rate limit information to add to the
`request` object.
Defaults to `rateLimit`.
### `store`
The storage to use when persisting rate limit attempts.
By default, the [memory store](source/memory-store.ts) is used.
Available data stores are:
- [memory-store](source/memory-store.ts): _(default)_ Simple in-memory option.
Does not share state when app has multiple processes or servers.
- [rate-limit-redis](https://npmjs.com/package/rate-limit-redis): A
[Redis](http://redis.io/)-backed store, more suitable for large or demanding
deployments.
- [rate-limit-memcached](https://npmjs.org/package/rate-limit-memcached): A
[Memcached](https://memcached.org/)-backed store.
- [rate-limit-mongo](https://www.npmjs.com/package/rate-limit-mongo): A
[MongoDB](https://www.mongodb.com/)-backed store.
- [precise-memory-rate-limit](https://www.npmjs.com/package/precise-memory-rate-limit) -
A memory store similar to the built-in one, except that it stores a distinct
timestamp for each IP rather than bucketing them together.
You may also create your own store. It must implement the `Store` interface as
follows:
```ts
import rateLimit, {
Store,
Options,
IncrementResponse,
} from 'express-rate-limit'
/**
* A {@link Store} that stores the hit count for each client.
*
* @public
*/
class SomeStore implements Store {
/**
* Some store-specific parameter.
*/
customParam!: string
/**
* The duration of time before which all hit counts are reset (in milliseconds).
*/
windowMs!: number
/**
* @constructor for {@link SomeStore}. Only required if the user needs to pass
* some store specific parameters. For example, in a Mongo Store, the user will
* need to pass the URI, username and password for the Mongo database.
*
* @param customParam {string} - Some store-specific parameter.
*/
constructor(customParam: string) {
this.customParam = customParam
}
/**
* Method that actually initializes the store. Must be synchronous.
*
* @param options {Options} - The options used to setup the middleware.
*
* @public
*/
init(options: Options): void {
this.windowMs = options.windowMs
// ...
}
/**
* Method to increment a client's hit counter.
*
* @param key {string} - The identifier for a client
*
* @returns {IncrementResponse} - The number of hits and reset time for that client
*
* @public
*/
async increment(key: string): Promise<IncrementResponse> {
// ...
return {
totalHits,
resetTime,
}
}
/**
* Method to decrement a client's hit counter.
*
* @param key {string} - The identifier for a client
*
* @public
*/
async decrement(key: string): Promise<void> {
// ...
}
/**
* Method to reset a client's hit counter.
*
* @param key {string} - The identifier for a client
*
* @public
*/
async resetKey(key: string): Promise<void> {
// ...
}
/**
* Method to reset everyone's hit counter.
*
* @public
*/
async resetAll(): Promise<void> {
// ...
}
}
export default SomeStore
```
## Instance API
### `resetKey(key)`
Resets the rate limiting for a given key. An example use case is to allow users
to complete a captcha or whatever to reset their rate limit, then call this
method.
## Issues and Contributing
If you encounter a bug or want to see something added/changed, please go ahead
and [open an issue](https://github.com/nfriedly/express-rate-limit/issues/new)!
If you need help with something, feel free to
[start a discussion](https://github.com/nfriedly/express-rate-limit/discussions/new)!
If you wish to contribute to the library, thanks! First, please read
[the contributing guide](contributing.md). Then you can pick up any issue and
fix/implement it!
## License
MIT © [Nathan Friedly](http://nfriedly.com/)

13
node_modules/express-rate-limit/tsconfig.json generated vendored Normal file
View File

@ -0,0 +1,13 @@
{
"include": ["source/"],
"exclude": ["node_modules/"],
"compilerOptions": {
"declaration": true,
"strict": true,
"noUnusedLocals": true,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true,
"esModuleInterop": true,
"moduleResolution": "node"
}
}

18
package-lock.json generated
View File

@ -11,6 +11,7 @@
"dependencies": {
"@mozilla/readability": "^0.3.0",
"express": "^4.17.1",
"express-rate-limit": "^6.0.5",
"jsdom": "^16.4.0",
"turndown": "^7.0.0"
},
@ -418,6 +419,17 @@
"node": ">= 0.10.0"
}
},
"node_modules/express-rate-limit": {
"version": "6.0.5",
"resolved": "https://registry.npmjs.org/express-rate-limit/-/express-rate-limit-6.0.5.tgz",
"integrity": "sha512-EB1mRTrzyyPfEsQZIQFXocd8NKZoDZbEwrtbdgkc20Yed6oYg02Xfjza2HHPI/0orp54BrFeHeT92ICB9ydokw==",
"engines": {
"node": ">= 14.5.0"
},
"peerDependencies": {
"express": "^4"
}
},
"node_modules/fast-levenshtein": {
"version": "2.0.6",
"resolved": "https://registry.npmjs.org/fast-levenshtein/-/fast-levenshtein-2.0.6.tgz",
@ -1433,6 +1445,12 @@
"vary": "~1.1.2"
}
},
"express-rate-limit": {
"version": "6.0.5",
"resolved": "https://registry.npmjs.org/express-rate-limit/-/express-rate-limit-6.0.5.tgz",
"integrity": "sha512-EB1mRTrzyyPfEsQZIQFXocd8NKZoDZbEwrtbdgkc20Yed6oYg02Xfjza2HHPI/0orp54BrFeHeT92ICB9ydokw==",
"requires": {}
},
"fast-levenshtein": {
"version": "2.0.6",
"resolved": "https://registry.npmjs.org/fast-levenshtein/-/fast-levenshtein-2.0.6.tgz",

2
package.json
View File

@ -6,10 +6,10 @@
"dependencies": {
"@mozilla/readability": "^0.3.0",
"express": "^4.17.1",
"express-rate-limit": "^6.0.5",
"jsdom": "^16.4.0",
"turndown": "^7.0.0"
},
"devDependencies": {},
"scripts": {
"start": "node index.js"
},