TimeTiles Documentation
GitHubGitHub (opens in a new tab)
  • Introduction
    • Overview
    • What is TimeTiles?
    • Quick Start
    • Installation Options
    • FAQ
      • Overview
      • Importing Your First Dataset
      • Creating Interactive Timelines
      • Using Filters and Search
      • Sharing and Embedding
      • Working with Different Data Formats
      • Overview
      • Timeline Navigation
      • Map Interactions
      • Data Visualization Options
      • Export Capabilities
      • Overview & Examples
      • Overview & Tech Stack
      • Data Processing Pipeline
      • Processing Stages & Jobs
      • Dynamic Filters
      • Development Setup
      • Testing Guidelines
      • Test Debugging Guide
      • Commit Guidelines
      • Overview
          • lib/collections/catalogs
            • Variable: default
          • lib/collections/dataset-schemas
            • Variable: default
          • lib/collections/datasets
            • Variable: default
          • lib/collections/events
            • Variable: default
          • lib/collections/geocoding-providers
            • Variable: GeocodingProviders
          • lib/collections/import-files
            • Variable: default
          • lib/collections/import-jobs
            • Variable: default
          • lib/collections/location-cache
            • Variable: default
          • lib/collections/media
            • Variable: default
          • lib/collections/pages
            • Variable: Pages
          • lib/collections/shared-fields
            • Function: createCommonConfig()
            • Function: createSlugField()
            • Interface: CommonCollectionOptions
            • Variable: basicMetadataFields
            • Variable: isEditorOrAdmin
            • Variable: metadataField
          • lib/collections/users
            • Variable: default
          • lib/services/error-recovery
            • Class: ErrorRecoveryService
            • Interface: ErrorClassification
            • Interface: RecoveryResult
            • Interface: RetryConfig
          • lib/services/geocoding
            • lib/services/geocoding/cache-manager
              • Class: CacheManager
            • Function: geocodeAddress()
            • Function: initializeGeocoding()
            • lib/services/geocoding/geocoding-operations
              • Class: GeocodingOperations
            • lib/services/geocoding/geocoding-service
              • Class: GeocodingService
            • lib/services/geocoding/provider-manager
              • Class: ProviderManager
            • lib/services/geocoding/types
              • Class: GeocodingError
              • Interface: BatchGeocodingResult
              • Interface: GeocodingResult
              • Interface: GeocodingSettings
              • Interface: ProviderConfig
              • Variable: LOCATION\_CACHE\_COLLECTION
              • Variable: NOMINATIM\_BASE\_URL
              • Variable: TIMETILES\_USER\_AGENT
          • lib/services/id-generation
            • Class: IdGenerationService
            • Function: generateUniqueId()
            • lib/services/import/coordinate-parser
              • Function: parseCoordinate()
              • Function: parseDMSFormat()
              • Function: parseDirectionalFormat()
              • Function: tryParseDecimal()
            • lib/services/import/coordinate-validation-utils
              • Function: categorizeCoordinateValue()
              • Function: isValidCoordinate()
              • Function: isValidLatitudeCandidate()
              • Function: isValidLongitudeCandidate()
              • Function: valueToString()
              • Interface: CoordinateSample
            • lib/services/import/coordinate-validator
              • Class: CoordinateValidator
              • Interface: CoordinateExtraction
              • Interface: ValidatedCoordinates
            • lib/services/import/format-detector
              • Function: checkCommaFormat()
              • Function: checkGeoJsonFormat()
              • Function: checkSpaceFormat()
              • Interface: FormatDetectionResult
              • Variable: combinedPatterns
              • Variable: latitudePatterns
              • Variable: longitudePatterns
            • lib/services/import/geo-location-detector
              • Class: GeoLocationDetector
              • Interface: GeoColumnResult
          • lib/services/progress-tracking
            • Class: ProgressTrackingService
            • Interface: DuplicationSummary
            • Interface: ProgressInfo
          • lib/services/rate-limit-service
            • Class: RateLimitService
            • Function: getClientIdentifier()
            • Function: getRateLimitService()
            • Variable: RATE\_LIMITS
          • lib/services/schema-builder
            • Class: ProgressiveSchemaBuilder
          • lib/services/schema-versioning
            • Class: SchemaVersioningService
          • lib/services/stage-transition
            • Class: StageTransitionService
            • Interface: StageTransitionResult
          • lib/services/type-transformation
            • Class: TypeTransformationService
            • lib/jobs/handlers/analyze-duplicates-job
              • Variable: analyzeDuplicatesJob
            • lib/jobs/handlers/cleanup-approval-locks-job
              • Variable: cleanupApprovalLocksJob
            • lib/jobs/handlers/create-events-batch-job
              • Variable: createEventsBatchJob
            • lib/jobs/handlers/create-schema-version-job
              • Variable: createSchemaVersionJob
            • lib/jobs/handlers/dataset-detection-job
              • Variable: datasetDetectionJob
            • lib/jobs/handlers/geocode-batch-job
              • Variable: geocodeBatchJob
            • lib/jobs/handlers/schema-detection-job
              • Variable: schemaDetectionJob
            • lib/jobs/handlers/validate-schema-job
              • Variable: validateSchemaJob
          • lib/jobs/import-jobs
            • lib/jobs/types/job-inputs
              • Interface: AnalyzeDuplicatesJobInput
              • Interface: BaseJobInput
              • Interface: BatchJobInput
              • Interface: CreateEventsBatchJobInput
              • Interface: CreateSchemaVersionJobInput
              • Interface: DatasetDetectionJobInput
              • Interface: FileJobInput
              • Interface: GeocodingBatchJobInput
              • Interface: ImportJobInput
              • Interface: SchemaDetectionJobInput
              • Interface: ValidateSchemaJobInput
            • lib/jobs/utils/data-parsing
              • Function: getObjectProperty()
              • Function: parseCSVFile()
              • Function: parseExcelFile()
              • Function: parseFileByType()
              • Function: setObjectProperty()
            • lib/jobs/utils/data-validation
              • Function: hasValidProperty()
              • Function: parseDate()
              • Function: parseTagsFromRow()
              • Function: safeStringValue()
              • Function: validateRequiredFields()
            • lib/jobs/utils/event-processing
              • Function: extractCoordinatesFromRow()
              • Function: processRowData()
            • lib/jobs/utils/job-context
              • Function: extractEventCreationContext()
              • Function: extractFileParsingContext()
              • Interface: BatchProcessingJobPayload
              • Interface: EventCreationJobPayload
              • Interface: FileParsingJobPayload
              • Interface: GeocodingBatchJobPayload
              • Type Alias: JobHandlerContext\<T\>
              • app/api/events/histogram/route
                • Function: GET()
              • app/api/events/list/route
                • Function: GET()
              • app/api/events/map-clusters/route
                • Function: GET()
            • app/api/events/route
              • Function: GET()
            • app/api/health/route
              • Function: GET()
          • [Importid]
            • app/api/preview/route
              • Function: GET()
          • lib/hooks/use-debounce
            • Function: useDebounce()
            • Function: useDebounceWithComparison()
          • lib/hooks/use-event-stats
            • Function: useEventDateAccessor()
            • Function: useEventStats()
            • Function: useEventsByCatalog()
            • Function: useEventsByDataset()
            • Interface: EventStats
          • lib/hooks/use-events-queries
            • Function: useEventsListQuery()
            • Function: useHistogramQuery()
            • Function: useImportProgressQuery()
            • Function: useImportUploadMutation()
            • Function: useInvalidateEventsQueries()
            • Function: useMapClustersQuery()
            • Interface: EventsListResponse
            • Interface: HistogramData
            • Interface: HistogramResponse
            • Interface: ImportJobProgress
            • Interface: ImportProgressResponse
            • Interface: MapClustersResponse
            • Interface: SimpleBounds
            • Variable: eventsQueryKeys
          • lib/hooks/use-theme
            • Function: useTheme()
          • lib/utils/date
            • Function: formatDate()
            • Function: formatDateShort()
          • lib/utils/file-readers
            • Function: getFileRowCount()
            • Function: readBatchFromFile()
          • lib/utils/slug
            • Function: createSlugHook()
            • Function: generateSlug()
            • Function: generateUniqueSlug()
          • lib/filters
            • Function: useFilters()
          • lib/health
            • Function: runHealthChecks()
            • Interface: HealthCheckResult
          • lib/logger
            • Function: createJobLogger()
            • Function: createLogger()
            • Function: createRequestLogger()
            • Function: logError()
            • Function: logPerformance()
            • Type Alias: LogLevelType
            • Variable: LogLevel
            • Variable: logger
          • lib/store
            • Function: clearAllFilters()
            • Function: getActiveFilterCount()
            • Function: hasActiveFilters()
            • Function: removeFilter()
            • Interface: FilterState
            • Variable: useUIStore
          • lib/constants/import-constants
            • Type Alias: CollectionName
            • Type Alias: ImportStatus
            • Type Alias: JobType
            • Type Alias: ProcessingStage
            • Variable: BATCH\_SIZES
            • Variable: COLLECTION\_NAMES
            • Variable: IMPORT\_STATUS
            • Variable: JOB\_TYPES
            • Variable: PROCESSING\_STAGE
          • lib/types/geocoding
            • Function: getGeocodingCandidate()
            • Function: getGeocodingResultForRow()
            • Function: getGeocodingResults()
            • Function: isValidGeocodingCandidate()
            • Function: isValidGeocodingResultsMap()
            • Interface: GeocodingCandidate
            • Interface: GeocodingResult
            • Interface: GeocodingResultsMap
          • lib/types/schema-detection
            • Function: getFieldStats()
            • Function: getSchemaBuilderState()
            • Function: isValidSchemaBuilderState()
            • Interface: DuplicateAction
            • Interface: DuplicateInfo
            • Interface: FieldStatistics
            • Interface: SchemaBuilderState
            • Interface: SchemaChange
            • Interface: SchemaComparison
            • Interface: SchemaField
          • lib/seed
              • lib/seed/core/seed-manager-base
                • Abstract Class: SeedManagerBase
            • lib/seed/database-operations
              • Class: DatabaseOperations
              • Function: createDatabaseOperations()
              • Interface: BatchOperationResult
              • lib/seed/operations/config-driven-seeding
                • Class: ConfigDrivenSeeding
              • lib/seed/operations/data-processing
                • Class: DataProcessing
              • lib/seed/operations/query-builders
                • Class: QueryBuilders
              • lib/seed/operations/seeding-operations
                • Class: SeedingOperations
              • lib/seed/operations/truncation
                • Class: TruncationOperations
            • lib/seed/relationship-config
              • Function: getCollectionsWithRelationships()
              • Function: getDependencyOrder()
              • Function: getRelationshipConfig()
              • Function: validateRelationshipConfig()
              • Interface: RelationshipConfig
              • Variable: RELATIONSHIP\_CONFIG
            • lib/seed/relationship-resolver
              • Class: RelationshipResolver
              • Interface: ResolutionStats
              • Interface: ResolvedItem
            • lib/seed/seed-manager
              • Class: SeedManager
              • Function: createSeedManager()
            • lib/seed/seed.config
              • Function: getCollectionConfig()
              • Function: getEnabledCollections()
              • Function: getEnvironmentSettings()
              • Function: getGeneratorConfig()
              • Interface: CollectionConfig
              • Interface: EnvironmentConfig
              • Interface: GeneratorConfig
              • Interface: SeedConfiguration
              • Variable: SEED\_CONFIG
              • lib/seed/seeds/catalogs
                • Function: catalogSeeds()
                • Type Alias: CatalogSeed
              • lib/seed/seeds/datasets
                • Function: datasetSeeds()
                • Type Alias: DatasetSeed
              • lib/seed/seeds/events
                • Function: eventSeeds()
                • Type Alias: EventSeed
              • lib/seed/seeds/import-files
                • Function: importFileSeeds()
                • Type Alias: ImportFileSeed
              • lib/seed/seeds/import-jobs
                • Function: importJobSeeds()
                • Type Alias: ImportJobSeed
              • lib/seed/seeds/main-menu
                • Variable: mainMenuSeed
              • lib/seed/seeds/pages
                • Variable: pagesSeed
              • lib/seed/seeds/users
                • Function: userSeeds()
                • Type Alias: UserSeed
              • lib/seed/seeds/utils
                • Function: generateCoordinate()
                • Function: generateMetadata()
                • Function: getDatasetsPerCatalog()
                • Function: getEventsPerDataset()
                • Function: getRegionForDataset()
                • Function: getSchemaTypeForCatalog()
                • Variable: DATASET\_SCHEMAS
                • Variable: GEOGRAPHIC\_REGIONS
            • lib/seed/types
              • Interface: SeedOptions
              • Type Alias: SeedData
            • lib/globals/main-menu
              • Variable: MainMenu
      • Readme
        • Api
        • Collections
            • lib/config/payload-config-factory
              • Function: createMinimalTestConfig()
              • Function: createPayloadConfig()
              • Function: createProductionConfig()
              • Function: createTestConfig()
              • Interface: PayloadConfigOptions
            • lib/config/payload-shared-config
              • Type Alias: CollectionName
              • Variable: ALL\_COLLECTIONS
              • Variable: ALL\_GLOBALS
              • Variable: ALL\_JOBS
              • Variable: COLLECTIONS
              • Variable: DEFAULT\_COLLECTIONS
              • Variable: DEFAULT\_DB\_CONFIG
              • Variable: DEFAULT\_TYPESCRIPT\_CONFIG
              • Variable: DEFAULT\_UPLOAD\_CONFIG
        • Constants
        • Filters
        • Globals
        • Health
        • Hooks
        • Jobs
        • Logger
        • Seed
        • Services
        • Store
        • Types
        • Utils
      • Index
    • Roadmap
    • Changelog
    • Glossary

On This Page

  • Classes
Question? Give us feedback → (opens in a new tab)Edit this page on GitHub →
Developer Docs
API Reference
Development Tools
seeding
Operations
Data Processing
lib/seed/operations/data-processing

web

web / lib/seed/operations/data-processing

lib/seed/operations/data-processing

Classes

  • DataProcessing
Class: ConfigDrivenSeedingClass: DataProcessing
TimeTiles Documentation