Skip to main content

datapipe-js/utils

datapipe-js/utils is a set of JavaScript utility and helper functions for parsing, converting many other data management tasks

parseCsv#

Parse Csv text into array of items. It automatically recognizes Dates, numbers and booleans. Also, provides number of other options to skip values converts etc

parseCsv(content: string, options?: ParsingOptions): ScalarObject[]

Parsing options:

NameDefinitionDefault Value
delimiterDefine delimiter','
skipRowsRows to skip0
textFieldsa list of text fields what requires NO conversion[]
dateFieldsa list of date fields (string or string[]) e.g. ['Date1', ['Date2', 'MM/dd/yyyy'], ['Date3', 'yyyyMM']][]
numberFieldsA list of fields what should be parsed as numbers or stay blank[]
booleanFieldsA list of boolean fields[]
skipUntilA callback function to provide skip initial rows. (tokens: string[]) => boolean
takeWhileA callback function to provide a take while rows (tokens: string[]) => boolean
elementSelectorelement definition clallback (fieldDescriptions: FieldDescription[], tokens: string[]) => any

Date and number fields will be converted automatically. In most cases you don't have to confugire fields. Please use textFields property if you don't want some particular field to be converted. Also, use dateFields to specify date format e.g. US date: MM/dd/yyyy.

parseCsvToTable#

Parse Csv text into tables of string values. It does not resolves any types, but it provides suggestions. All other features like skipWhile, take while works well there

parseCsvToTable(content: string, options?: ParsingOptions): StringsDataTable

toCsv#

Converts array to the delimiter (comma) separated value

toCsv(array: ScalarObject[], delimiter = ','): string

fromTable#

Converts rows and columns to a list of ScalarObjects.

  • rowsOrTable Table data or Array of values .
  • fieldNames Column names. If not provided then, it will be auto generated
  • fieldDataTypes Column names
fromTable(
rowsOrTable: PrimitiveType[][] | TableDto,
fieldNames?: string[],
fieldDataTypes?: DataTypeName[]
): ScalarObject[]

toTable#

Converts an array of ScalarObjects into the table. An efficient format to transfer over the wire

  • values an array of ScalarObjects
toTable(values: ScalarObject[]): TableDto

Takes an array

[
{f1: 11, f2: 12, f3: 13},
{f1: 21, f2: 22, f3: 23},
{f1: 31, f2: 32, f3: 33}
]

Converts to

{
fieldNames: ["f1", "f2", "f3"],
rows:[
[11, 12, 13],
[21, 22, 23],
[31, 32, 33]
]
}

toObject#

Converts array of items to the object map. Where key selector can be defined.

  • array array to be converted
  • keyField a key field selected
toObject(array: any[], keyField: string | string[] | Selector<any, string>): Record<string, any>

toSeries#

Convert array into an object of series.

const array = [
{f1: 11, f2: 12, f3: 13},
{f1: 21, f2: 22, f3: 23},
{f1: 31, f2: 32, f3: 33},
]
consoe.log(toSeries(object))
// output object
// {
// f1: [11, 21, 31],
// f2: [12, 22, 32],
// f3: [13, 23, 33]
//}

parseDatetimeOrNull#

Parses string to the Date or returns null. It is more efficient parser than new Date() and works better with UK date format

  • value value to parse
parseDatetimeOrNull(value: string | Date): Date | null

parseNumberOrNull#

Parses string to the number or returns null. It is more efficient parser than standard (parseFloat) and parses numbers like 1,000.32

  • value value to parse
parseNumberOrNull(value: string | Number): Number | null

parseBooleanOrNull#

Parses string to the boolean or returns null. It is treating ['1', 'yes', 'true', 'on'] as true and ['0', 'no', 'false', 'off'] as false

  • value value to parse
parseBooleanOrNull(value: string | Boolean): Boolean | null

dateToString#

Converts Date (or Date with time) to the ISO string with TimeZone trick. Usually it should be used before sending to the server

dateToString(d: Date): string

deepClone#

Returns a deep copy of your object or array.

deepClone(obj: any): any