pymaid

pymaid

pymaid (“python-catmaid”) lets you interface with a CATMAID server such as those provided by VFB to host published data from the FAFB dataset. It’s built on top of navis and returns generally returns data (neurons, volumes) in a way that you can plug them straight into navis - e.g. for plotting.

Connecting

The VFB servers (see here what’s available) are public and don’t require an API token for read-access which makes connecting dead simple:

import pymaid
import navis

navis.set_pbars(jupyter=False)
pymaid.set_pbars(jupyter=False)

# Connect to the VFB CATMAID server hosting the FAFB data
rm = pymaid.connect_catmaid(server="https://fafb.catmaid.virtualflybrain.org/", api_token=None, max_threads=10)

# Test call to see if connection works 
print(f'Server is running CATMAID version {rm.catmaid_version}')

WARNING: Could not load OpenGL library.
INFO  : Global CATMAID instance set. Caching is ON. (pymaid)
Server is running CATMAID version 2020.02.15-905-g93a969b37

We will cover how to search the VFB data base for neurons you might want to pull from the CATMAID server elsewhere. Instead, this notebook should give you a flavour of what kind of data you can pull and how to handle it.

Pulling neurons

Let’s start with pulling neurons:

# Pull a neuron by its ID (16) -> this happens to be a olfactory PN too 
n = pymaid.get_neurons(16)
n

See how this neuron’s type is “CatmaidNeuron”?

That’s because pymaid subclasses navis.TreeNeuron $\rightarrow$ pymaid.CatmaidNeuron and navis.NeuronList $\rightarrow$ pymaid.CatmaidNeuronList. The purpose of that is to add a bit of extra functionality (such as lazy loading of data) but both CatmaidNeuron and CatmaidNeuronList work as drop in replacements for their parent class.

Proof:

# Plot CatmaidNeuron with navis
navis.plot3d(n, width=1000, connectors=True, c='k')

get_neurons() returns neurons including their “connectors” - i.e. pre- (red) and postsynapses (blue). For this particular neuron, the published data comprehensively labels the axonal synapses but not the dendrites. Analogous to the nodes table, you can access the connectors like so:

n.connectors.head()

Let’s run a bigger example and pull all data published with Bates, Schlegel et al. 2020. For this, we will use “annotations”. These are effectively text labels that group neurons together.

bates = pymaid.find_neurons(annotations='Paper: Bates and Schlegel et al 2020')
len(bates)

INFO  : Found 583 neurons matching the search parameters (pymaid)





583

bates is a CatmaidNeuronList containing 583 neurons. Importantly pymaid has not yet loaded any data other than names! Note all the “NAs” in the summary:

bates.head()

We could have used pymaid.get_neurons('annotation:Paper: Bates and Schlegel et al 2020') instead to load all data up-front.

But: the free Deepnote machines are limited to 4Gb memory though and we might exceed that (soft) limit by loading all neurons at once - in particular if there are several notebooks running in parallel. Feel free to try it with get_neurons but keep an eye on the memory usage!

Continuing with our example: the CatmaidNeuronList will lazy load data from the server as you request it.

# Access the first neurons nodes 
# -> this will trigger a data download
_ = bates[0].nodes 

# Run summary again 
bates.head()

Note how the first neuron now has data where there were only NAs before? That’s because we loaded it on-demand.

Let’s do something more useful next: find and plot all uniglomelar DA1 projection neurons by their name.

# Name will be match pattern "Uniglomerular {tract} DA1 {lineage}"
import re 
prog = re.compile("Uniglomerular(.*?) DA1 ")

# Match all neuron names in `bates` against that pattern
is_da1 = list(map(lambda x: prog.match(x) != None, bates.name))

# Subset list 
da1 = bates[is_da1]
da1.head()

# Plot neurons by their lineage  
for n in da1:
    # Split name into components and keep only the tract
    n.lineage = n.name.split(' ')[3]    

# Generate a color per tract
import seaborn as sns
import numpy as np 

lineages = np.unique(da1.lineage) 
lin_cmap = dict(zip(lineages, sns.color_palette('muted', len(lineages))))
neuron_cmap = {n.id: lin_cmap[n.lineage] for n in da1}

navis.plot3d(da1, color=neuron_cmap, hover_name=True)

Let’s throw in the neuropil mesh for good measure. CATMAID servers can also host “volumes” (i.e. meshes). To find out what’s available:

vols = pymaid.get_volume()
vols.head()

INFO  : Retrieving list of available volumes. (pymaid)

# Get the neuropil volume 
v14neuropil = pymaid.get_volume('v14.neuropil')

# Make it slightly more transparent
v14neuropil.color = (.8, .8, .8, .3)

INFO  : Cached data used. Use `pymaid.clear_cache()` to clear. (pymaid)

# Plot with neuropil volume
navis.plot3d([da1, v14neuropil], color=neuron_cmap)

Suggested exercise:

• find all uni-glomerular projection neurons (name starts with Uniglomerular)
• calculate the number of pre-/postsynapses in the right lateral horn (LH) (use pymaid.get_volume and navis.in_volume)
• group them by glomerulus (nomenclature is Uniglomerular {tract} {glomerulus} {lineage} {meta data})
• plot LH pre- vs postsynapses in a scatter plot (e.g. using seaborn.scatterplot)

Pulling connectivity

Broadly speaking, CATMAID let’s you fetch connectivity data as either list of up- and downstream partners or as whole adjacency matrices.

# Pull downstream partners of DA1 PNs
da1_ds = pymaid.get_partners(da1,
                             threshold=3,  # anything with >= 3 synapses
                             directions=['outgoing']  # downstream partners only
                              )

# Result is a pandas DataFrame
da1_ds.head()

INFO  : Fetching connectivity table for 17 neurons (pymaid)
INFO  : Done. Found 0 pre-, 270 postsynaptic and 0 gap junction-connected neurons (pymaid)

Each row is a synaptic downstream partner of our query DA1 neurons. The columns to the left contain the synapses they receive from individual query neurons. For example 1811442 (first row) receives 30 synapses from the DA1 PN with ID 2863104.

# Get an adjacency matrix between all Bates, Schlegel et al. neurons
adj = pymaid.adjacency_matrix(bates)
adj.head()

# Plot a quick & dirty adjacency matrix
import seaborn as sns 

ax = sns.clustermap(adj, vmax=10, cmap='Greys')

/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/matrix.py:649: UserWarning:

Clustering large matrix with scipy. Installing `fastcluster` may give better performance.

We can also ask for where in space specific connections are made:

# Axo-axonic connections between two different types of DA1 PNs
cn = pymaid.get_connectors_between(2863104, 1811442)
cn.head()

# Visualize
points = np.vstack(cn.connector_loc)

navis.plot3d([da1.idx[[2863104, 1811442]],  # plot the two neurons
              points],  # plot the points of synaptic contacts as scatter 
              scatter_kws=dict(name="synaptic contacts")
              )