⚠ This page is served via a proxy. Original site: https://github.com
This service does not collect credentials or authentication data.
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/explanation/computation-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ class Segmentation(dj.Computed):
definition = """
-> Scan
---
num_cells : uint32
num_cells : int64
cell_masks : <blob@>
"""

Expand Down
20 changes: 10 additions & 10 deletions src/explanation/entity-integrity.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ attributes:
class Recording(dj.Manual):
definition = """
-> Session
recording_idx : uint16 # Recording number within session
recording_idx : int32 # Recording number within session
---
duration : float32 # seconds
"""
Expand All @@ -163,7 +163,7 @@ class Segmentation(dj.Computed):
definition = """
-> Scan # Depends on Scan
---
num_cells : uint32
num_cells : int64
"""
```

Expand Down Expand Up @@ -193,7 +193,7 @@ class Subject(dj.Manual):
class Session(dj.Manual):
definition = """
-> Subject # Inherits subject_id
session_idx : uint16 # NEW dimension: session_idx
session_idx : int32 # NEW dimension: session_idx
---
session_date : date
"""
Expand All @@ -202,7 +202,7 @@ class Session(dj.Manual):
class Trial(dj.Manual):
definition = """
-> Session # Inherits subject_id, session_idx
trial_idx : uint16 # NEW dimension: trial_idx
trial_idx : int32 # NEW dimension: trial_idx
---
outcome : enum('success', 'fail')
"""
Expand Down Expand Up @@ -245,7 +245,7 @@ class SessionSummary(dj.Computed):
definition = """
-> Session # PK = (subject_id, session_idx)
---
num_trials : uint32
num_trials : int64
accuracy : float32
"""
```
Expand All @@ -264,13 +264,13 @@ class Detection(dj.Computed):
-> Image # Inherits image_id
-> DetectionParams # Inherits params_id
---
num_blobs : uint32
num_blobs : int64
"""

class Blob(dj.Part):
definition = """
-> master # Inherits (image_id, params_id)
blob_idx : uint16 # NEW dimension within detection
blob_idx : int32 # NEW dimension within detection
---
x : float32
y : float32
Expand Down Expand Up @@ -339,9 +339,9 @@ experimental structure and ensures correct joins through semantic matching.
# Wrong: experiment_id alone isn't unique
class Trial(dj.Manual):
definition = """
experiment_id : uint32
experiment_id : int64
---
trial_number : uint16 # Should be part of key!
trial_number : int32 # Should be part of key!
result : float32
"""
```
Expand All @@ -352,7 +352,7 @@ class Trial(dj.Manual):
# Wrong: timestamp makes every row unique, losing entity semantics
class Measurement(dj.Manual):
definition = """
subject_id : uint32
subject_id : int64
timestamp : datetime(6) # Microsecond precision
---
value : float32
Expand Down
8 changes: 4 additions & 4 deletions src/explanation/type-system.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,10 +62,10 @@ Standardized, scientist-friendly types that work identically across backends.
| `int16` | 16-bit signed | -32,768 to 32,767 |
| `int32` | 32-bit signed | ±2 billion |
| `int64` | 64-bit signed | ±9 quintillion |
| `uint8` | 8-bit unsigned | 0 to 255 |
| `uint16` | 16-bit unsigned | 0 to 65,535 |
| `uint32` | 32-bit unsigned | 0 to 4 billion |
| `uint64` | 64-bit unsigned | 0 to 18 quintillion |
| `int16` | 8-bit unsigned | 0 to 255 |
| `int32` | 16-bit unsigned | 0 to 65,535 |
| `int64` | 32-bit unsigned | 0 to 4 billion |
| `int64` | 64-bit unsigned | 0 to 18 quintillion |
| `float32` | 32-bit float | ~7 significant digits |
| `float64` | 64-bit float | ~15 significant digits |
| `decimal(n,f)` | Fixed-point | Exact decimal |
Expand Down
8 changes: 4 additions & 4 deletions src/explanation/whats-new-2.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ If you're upgrading from legacy DataJoint, these changes require code updates:
| **Fetch API** | `table.fetch()` | `table.to_dicts()` or `.to_arrays()` |
| **Update** | `(table & key)._update('attr', val)` | `table.update1({**key, 'attr': val})` |
| **Join** | `table1 @ table2` | `table1 * table2` (with semantic check) |
| **Type syntax** | `longblob`, `int unsigned` | `<blob>`, `uint32` |
| **Type syntax** | `longblob`, `int unsigned` | `<blob>`, `int64` |
| **Jobs** | `~jobs` table | Per-table `~~table_name` |

See the [Migration Guide](../how-to/migrate-to-v20.md) for complete upgrade steps.
Expand Down Expand Up @@ -75,7 +75,7 @@ Legacy DataJoint overloaded MySQL types with implicit conversions:
| `longblob` | `<blob>` |
| `attach` | `<attach>` |
| `blob@store` | `<blob@store>` |
| `int unsigned` | `uint32` |
| `int unsigned` | `int64` |

### Three-Tier Architecture

Expand Down Expand Up @@ -207,7 +207,7 @@ This enables efficient access to large datasets stored in Zarr, HDF5, or custom
### Deprecated Features

- **AdaptedTypes**: Replaced by codec system (still works but migration recommended)
- **Native type syntax**: `int unsigned` → `uint32` (warnings on new tables)
- **Native type syntax**: `int unsigned` → `int64` (warnings on new tables)
- **Legacy external storage** (`blob@store`): Replaced by `<blob@store>` codec syntax

### Legacy Support
Expand Down Expand Up @@ -238,7 +238,7 @@ Upgrading from DataJoint 0.x is a **phased process** designed to minimize risk:
- **No database changes** — legacy clients still work

### Phase 2: Type Migration (Reversible)
- Update database column comments to use core types (`:uint32:`, `:<blob>:`)
- Update database column comments to use core types (`:int64:`, `:<blob>:`)
- Rebuild `~lineage` tables for semantic matching
- Update Python table definitions
- **Legacy clients still work** — only metadata changed
Expand Down
4 changes: 2 additions & 2 deletions src/how-to/choose-storage-type.md
Original file line number Diff line number Diff line change
Expand Up @@ -523,7 +523,7 @@ class SpikeWaveforms(dj.Computed):
"""Extracted spike shapes"""
definition = """
-> ContinuousData
unit_id : uint32
unit_id : int64
---
waveforms : <npy@> # 20 MB array, lazy load
"""
Expand Down Expand Up @@ -571,7 +571,7 @@ class FluorescenceTraces(dj.Computed):
"""Extracted time series"""
definition = """
-> SegmentedCells
cell_id : uint32
cell_id : int64
---
trace : <blob@> # 500 KB per cell, deduplicated
"""
Expand Down
16 changes: 8 additions & 8 deletions src/how-to/define-tables.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ class MyTable(dj.Manual):
```python
definition = """
subject_id : varchar(16) # Subject identifier
session_idx : uint16 # Session number
session_idx : int32 # Session number
---
...
"""
Expand Down Expand Up @@ -137,7 +137,7 @@ See [Design Primary Keys](design-primary-keys.md) for detailed guidance on key s
|------|-------------|
| `bool` | Boolean (true/false) |
| `int8`, `int16`, `int32`, `int64` | Signed integers |
| `uint8`, `uint16`, `uint32`, `uint64` | Unsigned integers |
| `int16`, `int32`, `int64`, `int64` | Unsigned integers |
| `float32`, `float64` | Floating point |
| `decimal(m,n)` | Fixed precision decimal |
| `varchar(n)` | Variable-length string |
Expand Down Expand Up @@ -196,7 +196,7 @@ Native types are flagged with a warning at declaration time but are allowed. Cor
class Session(dj.Manual):
definition = """
-> Subject # References Subject table
session_idx : uint16
session_idx : int32
---
session_date : date
"""
Expand Down Expand Up @@ -227,15 +227,15 @@ class TaskType(dj.Lookup):
class Session(dj.Manual):
definition = """
-> Subject
session_idx : uint16
session_idx : int32
---
session_date : date
"""

class Trial(dj.Part):
definition = """
-> master
trial_idx : uint16
trial_idx : int32
---
outcome : enum('hit', 'miss')
reaction_time : float32
Expand All @@ -250,7 +250,7 @@ class SessionStats(dj.Computed):
definition = """
-> Session
---
n_trials : uint32
n_trials : int64
hit_rate : float32
"""

Expand All @@ -270,7 +270,7 @@ Declare indexes at the end of the definition, after all attributes:
```python
definition = """
subject_id : varchar(16)
session_idx : uint16
session_idx : int32
---
session_date : date
experimenter : varchar(50)
Expand All @@ -288,7 +288,7 @@ Tables are declared in the database when the `@schema` decorator applies to the
@schema # Table is declared here
class Session(dj.Manual):
definition = """
session_id : uint16
session_id : int32
---
session_date : date
"""
Expand Down
12 changes: 6 additions & 6 deletions src/how-to/design-primary-keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Combine attributes when a single attribute isn't unique:
class Session(dj.Manual):
definition = """
-> Subject
session_idx : uint16 # Session number within subject
session_idx : int32 # Session number within subject
---
session_date : date
"""
Expand Down Expand Up @@ -100,7 +100,7 @@ Foreign keys above the `---` become part of the primary key:
class Trial(dj.Manual):
definition = """
-> Session # In primary key
trial_idx : uint16 # In primary key
trial_idx : int32 # In primary key
---
-> Stimulus # NOT in primary key
outcome : enum('hit', 'miss')
Expand All @@ -111,11 +111,11 @@ class Trial(dj.Manual):

### Keep Keys Small

Prefer `uint16` over `int64` when the range allows:
Prefer `int32` over `int64` when the range allows:

```python
# Good: Appropriate size
session_idx : uint16 # Max 65,535 sessions per subject
session_idx : int32 # Max 65,535 sessions per subject

# Avoid: Unnecessarily large
session_idx : int64 # Wastes space, slower joins
Expand Down Expand Up @@ -145,7 +145,7 @@ session_date : date

# Good: Add a sequence number
-> Subject
session_idx : uint16
session_idx : int32
---
session_date : date
```
Expand All @@ -172,7 +172,7 @@ Once a table has data, primary keys cannot be changed. Plan carefully:
class Scan(dj.Manual):
definition = """
-> Session
scan_idx : uint8 # Might need uint16 for high-throughput
scan_idx : int16 # Might need int32 for high-throughput
---
...
"""
Expand Down
2 changes: 1 addition & 1 deletion src/how-to/manage-pipeline-project.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ class Probe(dj.Lookup):
definition = """
probe_type : varchar(32)
---
num_channels : uint16
num_channels : int32
"""

@schema
Expand Down
Loading