pymaid

Overview

pymaid lets you interface with a CATMAID server. It’s built on top of navis and returns data (neurons, volumes) in a way that you can plug them straight into navis to use features such as plotting.

Official documentation here.

Connecting

The VFB CATMAID servers (see here for what’s available) are public and don’t require an API token for read-only access which makes connecting simple:

import pymaid
import navis

navis.set_pbars(jupyter=False)
pymaid.set_pbars(jupyter=False)

# Connect to the VFB CATMAID server hosting the FAFB data
rm = pymaid.connect_catmaid(server="https://fafb.catmaid.virtualflybrain.org/", api_token=None, max_threads=10)

# Test call to see if connection works 
print(f'Server is running CATMAID version {rm.catmaid_version}')

WARNING: Could not load OpenGL library.
INFO  : Global CATMAID instance set. Caching is ON. (pymaid)
Server is running CATMAID version 2020.02.15-905-g93a969b37

Retrieving neurons

Let’s start with pulling a neuron based on its ID:

# Find a neuron from its ID (16) -> this is an olfactory projection neuron
n = pymaid.get_neurons(16)
n

This neuron’s type is pymaid.CatmaidNeuron, which is a subclass of navis.TreeNeuron. The list version is pymaid.CatmaidNeuronList, which a subclass of navis.NeuronList. This adds a bit of extra functionality (such as lazy loading of data) and allows CatmaidNeuron and CatmaidNeuronList work as drop in replacements for their parent classes.

# Plot CatmaidNeuron with navis
navis.plot3d(n, width=1000, connectors=True, c='k')

get_neurons() returns neurons including their “connectors” - i.e. pre- (red) and postsynapses (blue). For this particular neuron, the published data comprehensively labels the axonal synapses but not the dendrites. Analogous to the nodes table, you can access the connectors like so:

n.connectors.head()

Let’s run a bigger example and pull all data published with Bates, Schlegel et al. 2020. For this, we will use “annotations”. These are effectively text labels that group neurons together, in this case by paper. Instead of get_neurons we can use find_neurons to avoid downloading unnecessary data.

bates = pymaid.find_neurons(annotations='Paper: Bates and Schlegel et al 2020')
len(bates)

INFO  : Found 583 neurons matching the search parameters (pymaid)





583

bates is a CatmaidNeuronList containing 583 neurons. Importantly pymaid has not yet loaded any data other than names! Note all the “NAs” in the summary:

bates.head()

We could have used pymaid.get_neurons(annotations='Paper: Bates and Schlegel et al 2020') instead to load all data up-front, but this would increase memory usage.

The CatmaidNeuronList we have created will lazy load data from the server when required.

# Access the first neuron's nodes 
# -> this will trigger a data download
_ = bates[0].nodes 

# Run summary again 
bates.head()

We have now loaded data for the first neuron.

Next we willl find and plot all uniglomelar DA1 projection neurons by their name.

# Name will be match pattern "Uniglomerular {tract} DA1 {lineage}"
import re 
prog = re.compile("Uniglomerular(.*?) DA1 ")

# Match all neuron names in `bates` against that pattern
is_da1 = list(map(lambda x: prog.match(x) != None, bates.name))

# Subset list 
da1 = bates[is_da1]
da1.head()

# Plot neurons by their lineage  
for n in da1:
    # Split name into components and keep the lineage
    n.lineage = n.name.split(' ')[3]    

# Generate a color per lineage
import seaborn as sns
import numpy as np 

lineages = np.unique(da1.lineage) 
lin_cmap = dict(zip(lineages, sns.color_palette('muted', len(lineages))))
neuron_cmap = {n.id: lin_cmap[n.lineage] for n in da1}

navis.plot3d(da1, color=neuron_cmap, hover_name=True)

Let’s add the neuropil meshes. These are called “volumes” on the CATMAID servers. To find out what’s available:

vols = pymaid.get_volume()
vols.head()

INFO  : Retrieving list of available volumes. (pymaid)

# Get the neuropil volume 
v14neuropil = pymaid.get_volume('v14.neuropil')

# Make it slightly more transparent
v14neuropil.color = (.8, .8, .8, .3)

INFO  : Cached data used. Use `pymaid.clear_cache()` to clear. (pymaid)

# Plot with neuropil volume
navis.plot3d([da1, v14neuropil], color=neuron_cmap)

Suggested exercises:

• find all uniglomerular projection neurons (name starts with Uniglomerular)
• calculate the number of pre-/post-synapses in the right lateral horn (LH) (use pymaid.get_volume and navis.in_volume)
• group the neurons by glomerulus based on label (nomenclature is Uniglomerular {tract} {glomerulus} {lineage} {metadata})
• plot LH pre- vs post-synapses in a scatter plot (e.g. using seaborn.scatterplot)

Pulling connectivity

CATMAID lets you fetch connectivity data either as a list of up- and downstream partners or as whole adjacency matrices.

# Pull downstream partners of DA1 PNs
da1_ds = pymaid.get_partners(da1,
                             threshold=3,  # anything with >= 3 synapses
                             directions=['outgoing']  # downstream partners only
                              )

# Result is a pandas DataFrame
da1_ds.head()

INFO  : Fetching connectivity table for 17 neurons (pymaid)
INFO  : Done. Found 0 pre-, 270 postsynaptic and 0 gap junction-connected neurons (pymaid)

Each row is a synaptic downstream partner of our query DA1 neurons. The columns to the left contain the synapses they receive from individual query neurons. For example 1811442 (first row) receives 30 synapses from the DA1 PN with ID 2863104.

# Get an adjacency matrix between all Bates, Schlegel et al. neurons
adj = pymaid.adjacency_matrix(bates)
adj.head()

# Plot a quick & dirty adjacency matrix
import seaborn as sns 

ax = sns.clustermap(adj, vmax=10, cmap='Greys')

/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/matrix.py:649: UserWarning:

Clustering large matrix with scipy. Installing `fastcluster` may give better performance.

We can also ask for where in space specific connections are made:

# Axo-axonic connections between two different types of DA1 PNs
cn = pymaid.get_connectors_between(2863104, 1811442)
cn.head()

# Visualize
points = np.vstack(cn.connector_loc)

navis.plot3d([da1.idx[[2863104, 1811442]],  # plot the two neurons
              points],  # plot the points of synaptic contacts as scatter 
              scatter_kws=dict(name="synaptic contacts")
              )