Package: DelayedArray 0.33.2

Hervé Pagès

DelayedArray: A unified framework for working transparently with on-disk and in-memory array-like datasets

Wrapping an array-like object (typically an on-disk object) in a DelayedArray object allows one to perform common array operations on it without loading the object in memory. In order to reduce memory usage and optimize performance, operations on the object are either delayed or executed using a block processing mechanism. Note that this also works on in-memory array-like objects like DataFrame objects (typically with Rle columns), Matrix objects, ordinary arrays and, data frames.

Authors:Hervé Pagès [aut, cre], Aaron Lun [ctb], Peter Hickey [ctb]

DelayedArray_0.33.2.tar.gz
DelayedArray_0.33.2.zip(r-4.5)DelayedArray_0.33.2.zip(r-4.4)DelayedArray_0.33.2.zip(r-4.3)
DelayedArray_0.33.2.tgz(r-4.4-x86_64)DelayedArray_0.33.2.tgz(r-4.4-arm64)DelayedArray_0.33.2.tgz(r-4.3-x86_64)DelayedArray_0.33.2.tgz(r-4.3-arm64)
DelayedArray_0.33.2.tar.gz(r-4.5-noble)DelayedArray_0.33.2.tar.gz(r-4.4-noble)
DelayedArray_0.33.2.tgz(r-4.4-emscripten)DelayedArray_0.33.2.tgz(r-4.3-emscripten)
DelayedArray.pdf |DelayedArray.html✨
DelayedArray/json (API)
NEWS

# Install 'DelayedArray' in R:
install.packages('DelayedArray', repos = c('https://mojaveazure.r-universe.dev', 'https://cloud.r-project.org'))

Peer review:

Bug tracker:https://github.com/bioconductor/delayedarray/issues

On BioConductor:DelayedArray-0.33.0(bioc 3.21)DelayedArray-0.32.0(bioc 3.20)

infrastructuredatarepresentationannotationgenomeannotationbioconductor-packagecore-package

12.86 score 25 stars 1.2k packages 554 scripts 160 exports 14 dependencies

Last updated 7 days agofrom:78b5888796. Checks:OK: 1 WARNING: 5 ERROR: 3. Indexed: no.

TargetResultDate
Doc / VignettesOKNov 14 2024
R-4.5-win-x86_64WARNINGNov 15 2024
R-4.5-linux-x86_64WARNINGNov 15 2024
R-4.4-win-x86_64WARNINGNov 15 2024
R-4.4-mac-x86_64WARNINGNov 15 2024
R-4.4-mac-aarch64WARNINGNov 14 2024
R-4.3-win-x86_64ERRORNov 15 2024
R-4.3-mac-x86_64ERRORNov 15 2024
R-4.3-mac-aarch64ERRORNov 15 2024

Exports:acbindapermaperm.DelayedArrayaperm.SparseArraySeedapplyarbindArbitraryArrayGridas.array.SparseArraySeedas.data.frameas.matrixas.matrix.SparseArraySeedAutoRealizationSinkbindROWSBLOCK_write_to_sinkblockApplyblockReducecbindchunkdimchunkGridclosecoercecolAutoGridcolMaxscolMeanscolMinscolRangescolsumcolSumscolVarsConstantArrayConstantArraySeedcontentIsPristinecurrentBlockIdcurrentViewportdbinomdefaultAutoGriddefaultMultAutoGridsdefaultSinkAutoGridDelayedArraydense2sparsedimsdlogisdnormdpoisdropDummyArrayGrideffectiveGridendextract_arrayextract_sparse_arrayget_type_sizegetAutoBlockLengthgetAutoBlockShapegetAutoBlockSizegetAutoBPPARAMgetAutoGridMakergetAutoRealizationBackendgreplgridApplygridReducegsubis_noopis_sparseisEmptyisLinearisPristinemakeCappedVolumeBoxmakeNindexFromArrayViewportmakeRegularArrayGridOfCappedLengthViewportsmatrixClassmeanmean.DelayedArraymean.SparseArraySeedmodify_seedsncharnetSubsetAndApermnew_DelayedArraynseednzdatanzindexnzwhichOLD_extract_sparse_arraypaste2pathpath<-pbinomplogispmax2pmin2pnormppoisqbinomqlogisqnormqpoisrange.DelayedArrayrange.SparseArraySeedrangesrbindread_blockread_sparse_blockrealizeregisteredRealizationBackendsRegularArrayGridRleArrayrowAutoGridrowMaxsrowMeansrowMinsrowRangesrowsumrowsum.DelayedMatrixrowSumsrowVarsscalescale.DelayedMatrixseedseed<-seedApplyset_grid_contextsetAutoBlockShapesetAutoBlockSizesetAutoBPPARAMsetAutoGridMakersetAutoRealizationBackendshowshowAsCellshowtreesimplifysinkApplysparse2denseSparseArraySeedsplitsplit.DelayedArraysplitAsListstartsubsummarysummary.DelayedAbindsummary.DelayedApermsummary.DelayedNaryIsoOpsummary.DelayedOpsummary.DelayedSetDimnamessummary.DelayedSubsetsummary.DelayedUnaryIsoOpStacksummary.DelayedUnaryIsoOpWithArgssupportedRealizationBackendssweepttabletolowertouppertypetype<-uniqueunique.DelayedArrayupdateObjectwhichwidthwrite_block

Dependencies:abindBiocGenericscrayongenericsIRangeslatticeMatrixMatrixGenericsmatrixStatsS4ArraysS4VectorsSparseArrayXVectorzlibbioc

Working with large arrays in R (slides from July 2017)

Rendered fromA-Working_with_large_arrays.Rnwusingknitr::knitron Nov 14 2024.

Last update: 2024-05-06
Started: 2024-05-06

Implementing A DelayedArray Backend

Hervé Pagès

Rendered fromB-Implementing_a_backend.Rmdusingknitr::rmarkdownon Nov 14 2024.

Last update: 2024-05-06
Started: 2024-05-06

A DelayedArray / HDF5Array update (slides from April 2021)

Rendered fromC-DelayedArray_HDF5Array_update.Rnwusingknitr::knitron Nov 14 2024.

Last update: 2024-05-06
Started: 2024-05-06

Readme and manuals

Help Manual

Help pageTopics
Control the geometry of automatic blocksAutoBlock-global-settings getAutoBlockLength getAutoBlockShape getAutoBlockSize get_type_size setAutoBlockShape setAutoBlockSize
Create automatic grids to use for block processing of array-like objectsAutoGrid colAutoGrid defaultAutoGrid defaultSinkAutoGrid getAutoGridMaker rowAutoGrid setAutoGridMaker
blockApply() and familyblock processing blockApply blockReduce block_processing currentBlockId currentViewport effectiveGrid getAutoBPPARAM gridApply gridReduce setAutoBPPARAM set_grid_context
A DelayedArray subclass that contains a constant valueclass:ConstantArray class:ConstantArraySeed class:ConstantMatrix coerce,ConstantArray,ConstantMatrix-method coerce,ConstantMatrix,ConstantArray-method ConstantArray ConstantArray-class ConstantArraySeed ConstantArraySeed-class ConstantMatrix ConstantMatrix-class DelayedArray,ConstantArraySeed-method extract_array,ConstantArraySeed-method extract_sparse_array,ConstantArraySeed-method is_sparse,ConstantArraySeed-method matrixClass,ConstantArray-method
DelayedAbind objectsclass:DelayedAbind DelayedAbind DelayedAbind-class dim,DelayedAbind-method dimnames,DelayedAbind-method extract_array,DelayedAbind-method extract_sparse_array,DelayedAbind-method is_noop,DelayedAbind-method is_sparse,DelayedAbind-method summary,DelayedAbind-method summary.DelayedAbind updateObject,SeedBinder-method
DelayedAperm objectsclass:DelayedAperm DelayedAperm DelayedAperm-class dim,DelayedAperm-method dimnames,DelayedAperm-method extract_array,DelayedAperm-method extract_sparse_array,DelayedAperm-method is_noop,DelayedAperm-method is_sparse,DelayedAperm-method summary,DelayedAperm-method summary.DelayedAperm updateObject,SeedDimPicker-method
DelayedArray objectsaperm,DelayedArray-method aperm.DelayedArray c,DelayedArray-method class:DelayedArray class:DelayedArray1 class:DelayedMatrix coerce,DelayedArray,COO_SparseArray-method coerce,DelayedArray,DelayedMatrix-method coerce,DelayedArray,SparseArraySeed-method coerce,DelayedMatrix,DelayedArray-method DelayedArray DelayedArray,ANY-method DelayedArray,DelayedArray-method DelayedArray,DelayedOp-method DelayedArray-class DelayedArray1 DelayedArray1-class DelayedMatrix DelayedMatrix-class dim,DelayedArray-method dim<-,DelayedArray-method dimnames,DelayedArray-method dimnames<-,DelayedArray,ANY-method drop,DelayedArray-method extract_array,DelayedArray-method matrixClass matrixClass,DelayedArray-method names,DelayedArray-method names<-,DelayedArray-method new_DelayedArray show,DelayedArray-method split,DelayedArray,ANY-method split.DelayedArray splitAsList,DelayedArray-method type updateObject,DelayedArray-method [,DelayedArray-method [<-,DelayedArray-method [[,DelayedArray-method
Statistical functions on DelayedArray objectsdbinom dbinom,DelayedArray-method DelayedArray-stats dlogis dlogis,DelayedArray-method dnorm dnorm,DelayedArray-method dpois dpois,DelayedArray-method pbinom pbinom,DelayedArray-method plogis plogis,DelayedArray-method pnorm pnorm,DelayedArray-method ppois ppois,DelayedArray-method qbinom qbinom,DelayedArray-method qlogis qlogis,DelayedArray-method qnorm qnorm,DelayedArray-method qpois qpois,DelayedArray-method
Common operations on DelayedArray objects!,DelayedArray-method +,DelayedArray,missing-method -,DelayedArray,missing-method acbind,DelayedArray-method anyNA,DelayedArray-method apply apply,DelayedArray-method arbind,DelayedArray-method bindROWS,DelayedArray-method cbind cbind,DelayedArray-method DelayedArray-utils grepl,ANY,DelayedArray-method gsub,ANY,ANY,DelayedArray-method is.finite,DelayedArray-method is.infinite,DelayedArray-method is.na,DelayedArray-method is.nan,DelayedArray-method lengths,DelayedArray-method log,DelayedArray-method mean mean,DelayedArray-method mean.DelayedArray nchar,DelayedArray-method nzwhich,DelayedArray-method paste2 paste2,DelayedArray,DelayedArray-method paste2,DelayedArray,vector-method paste2,vector,DelayedArray-method pmax2 pmax2,ANY,ANY-method pmax2,DelayedArray,DelayedArray-method pmax2,DelayedArray,vector-method pmax2,vector,DelayedArray-method pmin2 pmin2,ANY,ANY-method pmin2,DelayedArray,DelayedArray-method pmin2,DelayedArray,vector-method pmin2,vector,DelayedArray-method range range,DelayedArray-method range.DelayedArray rbind rbind,DelayedArray-method round,DelayedArray-method scale scale,DelayedMatrix-method scale.DelayedMatrix signif,DelayedArray-method sub,ANY,ANY,DelayedArray-method sweep sweep,DelayedArray-method table table,DelayedArray-method tolower,DelayedArray-method toupper,DelayedArray-method type<-,DelayedArray-method unique unique,DelayedArray-method unique.DelayedArray which,DelayedArray-method
DelayedMatrix multiplication and cross-product%*% %*%,ANY,DelayedMatrix-method %*%,DelayedMatrix,ANY-method %*%,DelayedMatrix,DelayedMatrix-method crossprod crossprod,ANY,DelayedMatrix-method crossprod,DelayedMatrix,ANY-method crossprod,DelayedMatrix,DelayedMatrix-method crossprod,DelayedMatrix,missing-method DelayedMatrix-mult tcrossprod tcrossprod,ANY,DelayedMatrix-method tcrossprod,DelayedMatrix,ANY-method tcrossprod,DelayedMatrix,DelayedMatrix-method tcrossprod,DelayedMatrix,missing-method
rowsum() and colsum() on a DelayedMatrix objectcolsum colsum,DelayedMatrix-method DelayedMatrix-rowsum rowsum rowsum,DelayedMatrix-method rowsum.DelayedMatrix
DelayedNaryIsoOp objectsclass:DelayedNaryIsoOp DelayedNaryIsoOp DelayedNaryIsoOp-class dim,DelayedNaryIsoOp-method dimnames,DelayedNaryIsoOp-method extract_array,DelayedNaryIsoOp-method extract_sparse_array,DelayedNaryIsoOp-method is_sparse,DelayedNaryIsoOp-method summary,DelayedNaryIsoOp-method summary.DelayedNaryIsoOp updateObject,ConformableSeedCombiner-method
DelayedOp objectsclass:DelayedNaryOp class:DelayedOp class:DelayedUnaryIsoOp class:DelayedUnaryOp DelayedNaryOp DelayedNaryOp-class DelayedOp DelayedOp-class DelayedUnaryIsoOp DelayedUnaryIsoOp-class DelayedUnaryOp DelayedUnaryOp-class dim,DelayedUnaryIsoOp-method dimnames,DelayedUnaryIsoOp-method extract_array,DelayedUnaryIsoOp-method extract_sparse_array,DelayedUnaryIsoOp-method is_noop is_sparse,DelayedUnaryIsoOp-method summary,DelayedOp-method summary.DelayedOp updateObject,DelayedOp-method
DelayedSetDimnames objectsclass:DelayedSetDimnames DelayedSetDimnames DelayedSetDimnames-class dimnames,DelayedSetDimnames-method is_noop,DelayedSetDimnames-method summary,DelayedSetDimnames-method summary.DelayedSetDimnames updateObject,DelayedDimnames-method
DelayedSubassign objectsclass:DelayedSubassign DelayedSubassign DelayedSubassign-class extract_array,DelayedSubassign-method extract_sparse_array,DelayedSubassign-method is_noop,DelayedSubassign-method is_sparse,DelayedSubassign-method summary,DelayedSubassign-method summary.DelayedSubassign
DelayedSubset objectsclass:DelayedSubset DelayedSubset DelayedSubset-class dim,DelayedSubset-method dimnames,DelayedSubset-method extract_array,DelayedSubset-method extract_sparse_array,DelayedSubset-method is_noop,DelayedSubset-method is_sparse,DelayedSubset-method summary,DelayedSubset-method summary.DelayedSubset
DelayedUnaryIsoOpStack objectsclass:DelayedUnaryIsoOpStack DelayedUnaryIsoOpStack DelayedUnaryIsoOpStack-class extract_array,DelayedUnaryIsoOpStack-method extract_sparse_array,DelayedUnaryIsoOpStack-method is_sparse,DelayedUnaryIsoOpStack-method summary,DelayedUnaryIsoOpStack-method summary.DelayedUnaryIsoOpStack
DelayedUnaryIsoOpWithArgs objectsclass:DelayedUnaryIsoOpWithArgs DelayedUnaryIsoOpWithArgs DelayedUnaryIsoOpWithArgs-class extract_array,DelayedUnaryIsoOpWithArgs-method extract_sparse_array,DelayedUnaryIsoOpWithArgs-method is_sparse,DelayedUnaryIsoOpWithArgs-method summary,DelayedUnaryIsoOpWithArgs-method summary.DelayedUnaryIsoOpWithArgs
Utilities to make capped volume boxescapped_volume_boxes isLinear isLinear,ArrayGrid-method isLinear,ArrayViewport-method makeCappedVolumeBox makeRegularArrayGridOfCappedLengthViewports
DelayedMatrix row/col summarizationcolMaxs colMaxs,DelayedMatrix-method colMeans colMeans,DelayedMatrix-method colMins colMins,DelayedMatrix-method colRanges colRanges,DelayedMatrix-method colSums colSums,DelayedMatrix-method colVars colVars,DelayedMatrix-method matrixStats-methods rowMaxs rowMaxs,DelayedMatrix-method rowMeans rowMeans,DelayedMatrix-method rowMins rowMins,DelayedMatrix-method rowRanges rowRanges,DelayedMatrix-method rowSums rowSums,DelayedMatrix-method rowVars rowVars,DelayedMatrix-method
RealizationSink objectsarrayRealizationSink-class AutoRealizationSink class:arrayRealizationSink class:RealizationSink close,RealizationSink-method coerce,arrayRealizationSink,DelayedArray-method dim,arrayRealizationSink-method getAutoRealizationBackend RealizationSink RealizationSink-class registeredRealizationBackends setAutoRealizationBackend sinkApply supportedRealizationBackends write_block,arrayRealizationSink-method
Realize an object in memory or on diskBLOCK_write_to_sink realize realize,ANY-method
RleArray objectsclass:RleArray class:RleMatrix coerce,ANY,RleArray-method coerce,ANY,RleMatrix-method coerce,DataFrame,RleArray-method coerce,DelayedArray,RleArray-method coerce,DelayedMatrix,DataFrame-method coerce,DelayedMatrix,RleMatrix-method coerce,RleArray,Rle-method coerce,RleArray,RleMatrix-method coerce,RleList,RleArray-method coerce,RleMatrix,DataFrame-method coerce,RleMatrix,RleArray-method coerce,RleMatrix,RleList-method coerce,RleRealizationSink,DelayedArray-method coerce,RleRealizationSink,RleArray-method DelayedArray,RleArraySeed-method matrixClass,RleArray-method RleArray RleArray-class RleMatrix RleMatrix-class write_block,RleRealizationSink-method
RleArraySeed objectsChunkedRleArraySeed ChunkedRleArraySeed-class class:ChunkedRleArraySeed class:RleArraySeed class:RleRealizationSink class:SolidRleArraySeed coerce,ChunkedRleArraySeed,SolidRleArraySeed-method coerce,RleRealizationSink,ChunkedRleArraySeed-method coerce,RleRealizationSink,Rle-method coerce,RleRealizationSink,RleList-method coerce,SolidRleArraySeed,Rle-method dim,RleArraySeed-method dimnames,RleArraySeed-method extract_array,ChunkedRleArraySeed-method extract_array,SolidRleArraySeed-method RleArraySeed RleArraySeed-class RleRealizationSink RleRealizationSink-class SolidRleArraySeed SolidRleArraySeed-class type,RleRealizationSink-method
Visualize and access the leaves of a tree of delayed operationsmodify_seeds nseed nseed,ANY-method path path,DelayedOp-method path<-,DelayedOp-method seed seed,DelayedOp-method seed<- seed<-,DelayedOp-method seedApply show,DelayedOp-method showtree
Simplify a tree of delayed operationscontentIsPristine isPristine netSubsetAndAperm netSubsetAndAperm,ANY-method netSubsetAndAperm,DelayedArray-method simplify simplify,ANY-method simplify,DelayedAbind-method simplify,DelayedAperm-method simplify,DelayedArray-method simplify,DelayedNaryIsoOp-method simplify,DelayedSetDimnames-method simplify,DelayedSubassign-method simplify,DelayedSubset-method simplify,DelayedUnaryIsoOpStack-method simplify,DelayedUnaryIsoOpWithArgs-method
SparseArraySeed objectsaperm,SparseArraySeed-method aperm.SparseArraySeed as.array,SparseArraySeed-method as.array.SparseArraySeed as.matrix,SparseArraySeed-method as.matrix.SparseArraySeed class:SparseArraySeed coerce,ANY,SparseArraySeed-method coerce,SparseArraySeed,COO_SparseArray-method coerce,SparseArraySeed,CsparseMatrix-method coerce,SparseArraySeed,dgCMatrix-method coerce,SparseArraySeed,dgRMatrix-method coerce,SparseArraySeed,lgCMatrix-method coerce,SparseArraySeed,lgRMatrix-method coerce,SparseArraySeed,RsparseMatrix-method coerce,SparseArraySeed,sparseMatrix-method coerce,sparseMatrix,SparseArraySeed-method dense2sparse dim,SparseArraySeed-method dimnames,SparseArraySeed-method dimnames<-,SparseArraySeed,ANY-method extract_array,SparseArraySeed-method is_sparse,SparseArraySeed-method nzdata,SparseArraySeed-method nzindex OLD_extract_sparse_array sparse2dense SparseArraySeed SparseArraySeed-class
Operate natively on SparseArraySeed objectsall,SparseArraySeed-method any,SparseArraySeed-method anyNA,SparseArraySeed-method cbind,SparseArraySeed-method is.infinite,SparseArraySeed-method is.na,SparseArraySeed-method is.nan,SparseArraySeed-method max,SparseArraySeed-method mean,SparseArraySeed-method mean.SparseArraySeed min,SparseArraySeed-method nchar,SparseArraySeed-method prod,SparseArraySeed-method range,SparseArraySeed-method range.SparseArraySeed rbind,SparseArraySeed-method SparseArraySeed-utils sum,SparseArraySeed-method tolower,SparseArraySeed-method toupper,SparseArraySeed-method which,SparseArraySeed-method