YouTube Transcript:
Getting to Know NF-Core (Open Source Project Tours)

Skip watching entire videos - get the full transcript, search for keywords, and copy with one click.

Video Transcript

Video Summary

Summary

Core Theme

This content introduces the NF core open-source project, focusing on its structure, collaborative ethos, and commitment to reproducibility in bioinformatics workflows. It explores how the project facilitates community interaction and the technical underpinnings of its workflow management system.

Hi there. Uh, I'm Shaa Gord McKan. Um,

and I'm here today for the very first

ever um, open source project tour. Um,

so let me just do a little bit of an

intro to what I'm envisioning these

project tours to be with the caveat that

because this is the first one, I kind of

have no idea. Um but the general goal of

the project tours is to really explore

how open source projects are structured

um specifically with an eye to how they

facilitate collaboration or don't

facilitate collaboration but basically

thinking about open source projects as

sort of living communities of people who

are interacting with code with

documentation with other things. um and

just seeing how the structure of the

projects as a whole um meets the goals

of the project or some in some cases

doesn't. Um

so we're going to start with a bit of a

discussion about um today's project

which is called NF core and like the

history and goals of the project. We'll

talk a little bit about the different

stakeholders involved and the specific

spaces that the project lives in online

um or maybe offline, but we can only

explore the ones that are online. Um

maybe someday we can do a project tour

that involves like going to a conference

or something. Um but yeah, and then

we'll go into the tour itself um and

then do a little bit of a wrap-up at the

end. So that's the plan. We'll see how

it goes. Uh so yeah, let's get started

with an intro. So um do you want to

introduce yourself and NF core?

>> Uh yes. So hello my name is Maxim

Garcia. I'm a bioinformatician. I've

been working with Nexflow for almost 10

years now and I've been developing

Nexflow pipeline. So first at uh at the

Caroliska Institute here in Sweden and

then at SEAR who is the company that was

built on by the that was created by the

developers that created Nexflow

and Nexflow is a workflow manager that

really simplifies the way that we write

analysis pipeline and like we and a

whole community was built around that

which is NFore.

Um so what kind of pipelines? So I know

nothing about this project which I think

is maybe for anyone watching this who

also doesn't know anything about the

project because I will ask very simple

questions like um what kind of pipelines

what kind of workflows like is there

like a specific programming language

domain area that this is sort of arising

out of. Uh so at the beginning it was

mainly uh so at the beginning it's more bioinformatics

bioinformatics

>> specific to bionformatic is more

targeted toward genomics. So basically

analyzing DNA RNA or anything that goes

outside of a sequencer.

>> Gotcha. And so was it coming out of

academia or uh like a private lab or

>> so the whole history of the project

started uh actually the people behind

next were organizing events for like

other people that were developing like

that were working with next that they

can meet and like exchange ideas and uh

Phil Yul who has been working with

MulticQ and like other opensource first

project uh met uh two guys in in

Barcelona like at one of this event uh

Andreas and Alex and they decided that

okay we are doing like some stuff that

are pretty similar we are working on

like the same kind of data we have the

same kind of pipeline and um in the end

they decided okay it will be much easier

for everyone if we just join forces and

So very simply, so Phil came back from

Barcelona. He I was sitting next to him

at the time and he just asked me, "Oh,

Maxim, by the way, there's like this guy

in Germany like he wants to collaborate

on some pipeline." So let's let's get

you like and that's how I started

working onto that.

And uh yeah, and so that's already how

we started everything. So you were

involved pretty from pretty early on

from like about

>> Yeah, I was involved I think like before

the community was even started but like

I was not part of the original like team

that created the idea. So I'm just like

I'm not one of the founder of NFor but

I'm part of the building blocks I think.

>> Gotcha. And that was about 10 years ago.

Uh so no I started developing uh with

next flow in 2016. I think NGI which is

the the sequencing platform that Phil

was sitting at started developing

pipeline a couple of maybe a couple of

year priors to that and NFore started

like the idea was started like in late

2017 and we really started uh to do some

proper stuff in 2018. So

>> yeah, seven years ago more or less.

>> Okay. So it sounds like there was this

original idea to um

the sort of

let me try to repeat back to you and you

can correct the parts that I've

inevitably gotten wrong. So you um the

founders of the project were sort of

separately working on pipelines. They

get together, they decide, oh, let's

collaborate on pipelines. Um,

and you get looped in early on. Um, and

then there's like a couple years where

they're working on that and then it

turns into is like next flow um

a separate project or like a different

iteration of the project like what's the distinction?

distinction?

>> Uh, so so next flow is the the pro the

programming language that is like the

workflow manager that we are all using

to create to write our pipeline with. So

that will be the programming language

that we are all using and basically it

allow us to um to interface with every

um executors that we want so that people

can work uh locally on their laptop and

develop their pipeline like quite

easily. It can work on your own HPC

cluster at your university. It can work

on the cloud like AWS, Microsoft Azure

or Google Life Science and it's pretty

simple. It allow so necro allows you to

work with cond docker singularity or any

other kind of virtual environment or

like containerization that you would

want. So that way we are sure that

whatever is doing is reproducible

because for me that's one of the key

point into doing science.

If no one's can reproduce it, like what

are we doing?

>> Amazing. Yeah. No. Um I have a

background in the sciences and so in

psychology which has had some real

extreme issues with reproducibility. So

I totally get the importance of that. Um

cool. Okay. So um just checking in on my

questions. Uh

>> oh yeah and uh before we begin the name

NF core come from NF from next pretty

like pretty simple there and core it was

because it was started by core

facilities so sequencing platform that

were just doing like basic analysis

pipeline and all of the pipelines that

we had at NGI at the time started as

just like uh pipeline for quality

control that's just piled on and like

doing a bit more of analysis because for

us it was just simpler to do stuff. >> Yeah.

>> Yeah.

>> And that way we help kickstart for our

users their their analysis.

>> Um what does NGI stand for? uh NGI

that's for national genomic

infrastructure and that's one of the

multiple infrastructure within SC life

lab science for life laboratory which is

the main uh support of research here in Sweden.

Sweden.

>> Excellent. Um

cool. What would you say? Um

it sounds like reproducibility is a big

goal for the project. What are some

other sort of like high level goals for

the project?

>> Uh for me the main stuff is

reproducibly. Second stuff will be

collaboration because we like it all

started by collaborating and by not

trying to do the same stuff like like

like the same stuff at at different

location really trying to collaborate

together to work and to pilon and so for

me yes robot will be the the main stuff.

collaboration will be the second the

second main idea into into NF.

>> And who would you say are sort of the

different stakeholders in the project

whether that's users, contributors, funders,

funders,

um and also like with a broad brush like

because you could say people who benefit

from bioinformatics research are in some

sense a stakeholder.

>> Yeah. Uh so for me I would say like it's

uh so I would say like the people that

are interested in this project are the

people that are using this project. So

many that will be like either

researchers or uh sequencing platform or

uh even some clinician are using are

using some of the pipelines that we're

writing or even

>> what was the third you said researcher

frequency platforms and

>> clinicians like medical doctors that

clinicians clinical setting yeah

clinicians sorry my pronunciation is bad

>> much better than my pronunciation being

in any other language so No worries.

>> Uh I think uh we so we decided when we

when we when we launched NF core to have

like all of our pipeline work in MIT

license that way if people want to uh commercialize

commercialize

uh just running the pipeline they could.

So in such a sense uh even some like

private company are running NF core

pipeline sometimes participating into it

sometimes not but we are fully aware of

that and we embrace that like fully so I

would say even some big pharma company

are like interested into core and there

are like some collaboration on going

>> do you have a sense of how the companies

that are using NF core like how they're

using it

>> uh personally Not much because like it's

all on GitHub. So every pipeline that we

have that is public, we know how many

times that it's been forked or cloned.

But apart from that, we don't know. Uh I

know for sure that there is a lot of

dark stuff happening that we are not

aware of. But

I think it's good.

>> Yeah. Um that's something that's come up

in some of the projects that I've um

participated in or maintained. the sort

of like

great we know how many downloads we have

but like we don't know anything about

what anyone is doing with those

downloads and like the sort of like

black hole sometimes that it feels like

um you can have towards users. Um

cool. So you said um uh what was it

researchers platforms clinicians

some companies that are using it. Um

what I guess I have sort of two

different questions. What of those

stakeholders do you think is like the

largest group and then what stakeholders

would you say you interact with the most

or have the most direct impact on you?

>> Uh so for me personally I interact

mostly with uh either the de the people

that develop the pipeline. So that will

be mainly like the researcher or the

maintainer of the pipeline. So people

like me more or less and I do interact

quite a lot with users and definitely I

would say like user are the main uh

group of people.

>> Cool. Um are there any other

stakeholders um that we haven't talked about?

I think we did cover like um

we have re we have research we have

researchers or research group uh

people that are using it could be like

sequencing platform or labs sometimes

even university depending on how how

it's layered like in like in the

different countries and stuff like that.

Uh probably maybe some governmental

agencies that might be using that like

yeah but I think it's pretty broad and

like a lot of stuff like usually we know

when stuff are not working because

people came back to us but we don't know

where when stuff are working because

sometime like sometime people came back

to us saying okay what you've done is

great and we're using it sometime we

just don't know.

>> Yeah. Yeah. Um, are there any tensions

between stakeholders? Not necessarily in

terms of like active drama, but just

like places where the needs or desires

of different groups of stakeholders like

might conflict.

>> Uh, no, I haven't noticed that. uh like

I I did notice usually the the main

issue I think we have in term of like uh

conflict will be like uh trying to come

up with some decision at some at some

point uh and getting people to try to

collaborate together. We have like one

uh major rule in NF core which is

collaborate like because collaboration

mantra let's say and uh trying to get

people to collaborate could be sometime

difficult uh especially in clinical

settings where people have some like

specific logic that they need to to use

in their pipeline like locally and that

it's hard to collaborate with someone

that has like a different set of logic

that is

imposed onto them.

>> Gotcha. So do you think um well that

sounds a bit like um

my understanding of how you're

describing that is like a sort of like a

technical conflict in the sense of like

if some people need certain kinds of

logic and other people needs other kinds

of logic. So is it like a trying to

decide? I'm curious about whether we

this is something to discuss when we get

to like looking at like um the

repository versus

>> uh uh no for for me it's more a question

about like uh let's say like let's say

we we have like one infrastruct like a

certain set of tool another

infrastructure need to to use like

another set of tools and trying to put

like both set of tool like together

could be something that could be difficult.

difficult.

>> Gotcha. Yeah. Okay. And it isn't always

trivially easy to be like, oh, we'll

design a thing so that people can decide

which of the tools they use because like

designing like an interface or whatever

can be sometimes it's easy, but

sometimes it's actually way more

complicated than just designing.

>> Yeah. So

>> yeah, one last stuff that I think is

important all of the stuff that we are

producing is in common line.

So there are some company like Sakara

that can provide like a graphical

interface for for the for for this kind

of thing but most of the stuff most of

the work that we do is just on the

command line side.

>> Gotcha. Cool. Um

so one last question before we dive into

the tour itself um which is around

spaces. So like what kind of spaces does

the project exist in? and feel free to

name things that we can't actually

explore like oh we get together like

once a year at this particular

conference or we have our own convening

or whatever um so any sort of space

where people in the project are interacting

interacting

>> okay so I will start with the inerson

meeting so uh let's say like first to

explore more the the organization itself

so uh we're so we have a steering

committee of like I think it's five

people at the moment uh uh I I could go

like deeper into that that help us like

drive like more or less like the

direction where we we are going. We a

group of admins that is uh I think 12 to

15 people that really try to

administrate the project like daytoday

and solve all of the issue and figure

out like how to help people, how to

decide stuff and things like that. Um

we have three people that are like our

safety group that are there whenever

there is an issue and they can they

solve it and they don't reply to us they

directly reply to the steering

committee. So I think that's good

because like I think it's important for

a community to be aware that stuff can

happen badly and if that happen like

stuff are made to make sure that it's uh

it's handled well.

Uh we have one group that is dedicated

to outreach so and all of our social

network and helping organizing the event

that we organize.

Uh one group of mainteners uh that is

slightly bigger that help maintaining

like the world community and then

everyone else is a contributor.

So uh we take advantage of u

the next community quite a lot. So next

exist uh on the community side mainly on

slack with the next slack and on

discourse which is hosted by Sakara on

community.io IO which is basically a

forum that is great for asking question

and they also organize a summit so once

a year in Europe and this year it will

be uh virtual

and once a year in the US and it will

probably be in Boston again. uh

just prior to the summit uh there is a

hackathon that that is organized and

that's where people can meet up together

and work together on improving whatever

we are doing with NFore or we help

people working on their own project

using the NF core framework or tool

tooling that we that we have developed.

>> How long is that? Is that like a day or

a week or

>> uh so usually a summit is for is for

like two to three days and theaton would

be like for two days and in parallel of

the acon there is a team of people from

SERA that is organizing some training as well.

well.

>> Gotcha. And this is like a stand event.

Uh it's a it's a yes it's a it's a

conference and I think in term of people

there should be like 100 people for the

acaton more or less and something

similar for the training and I do think

it should be like way more for the for

the actual summit but yes this year

since it will be like virtual I'm a bit

I'm a bit unaware of the of the of the

numbers that it will be uh we also

organize a distributed hackathon

uh all all over the the world that like

people can contribute all together and

we try to organize that nicely. Uh so I

think in person that's that. Oh yes and

also every

15 to 18 months uh all of the admin try

to get together so that we can discuss

and we can like do stuff together and

like sync on some stuff and we just met

uh three weeks ago here in Stocken.

>> Cool. Cool. Um do you find that uh

do you find that um having that sort of

like focus time um is helpful because it

seems like if you're also like putting

on like a big summit with a hackathon

and trainings and all of these things

and having a sort of separate time to

like get heads down. Um

>> yes no definitely that's useful. We we

talk like quite to each other. So uh on

NFore we have Slack and we that's our

main uh network of communication and uh

we have like multiple channel all over

the place and we have like one channel

for us like administrator and we talk

with each other there like quite a lot

and but I do think that yes we've like

since co we know that how great it can

be like to work remotely to communicate

to each other like via Zoom and things

like that and it's great but sometimes

you feel that yes nothing can beat like

discussing like face to face with

someone else.

>> Yeah. Yeah. I guess I was thinking about

that. It just always looking out for

patterns for what seems to work for

projects. Um and it sounds like um

Nexflow like um NF core is like a

relatively larger project as far as open

source projects go. And I know for

Python like the Python language at a

certain point for a while they were

doing a lot of like language work at

Pyon like the big uh flagship conference

and at a certain point they were like

actually let's just let's just make it a

separate thing because the conference

itself becomes so much that having a

sort of separate space to just where

you're just focused on talking through

technical things or like working through

um working through whatever issues or or

conflicts. And again, when I use

conflict, I don't necessarily mean like

drama of people shouting at each other,

just like when people have two different

needs and they have to work out how to

get those needs met. Um, so yeah. Um,

yeah, I think this may be like a life

cycle going on cuz I think for smaller

projects it's actually common and I

asked a question about this um, which

was like I think for many projects you

start out colllocating at someone else's

conference. You're like, "Oh, we're all

going to be at PYON. we're all going to

be at this like scientific conference.

Um why don't we during the existing

hackathon or sprint or maybe come a day

before and just meet up. Um and then as

projects get bigger they start doing

their own thing and then projects might

get so bigger that their own thing is

actually not the best space for them to

get work done and it's more a space for

like the community as a whole. So it's

like the life cycle of projects as they

get bigger and bigger.

>> Yeah. I think at the moment we are still

like very tied to to next row like

the ties be between like Nexflow NFC

like are pretty like uh strong like a

lot of like obviously uh but yes so I

think at the moment we are still like

pretty tight to that but uh I think

years ago we were doing the training

like at NF core nowadays the training is

handled like by by Sakara so that's one

less thing that we're doing but

definitely what we're doing that is

distributed that is like completely

disconnected from like anything like

from the main next conference are the

distributed acaton. We still have local

acathaton that are like tied into the

conferences where we just have a smaller

group of people focused but when we do

our global distributed aaton we try to

get local group to self-organize and to

have a katon. So I think we are getting

towards that direction and definitely I

agree like it's such such like it's what

happened in the life cycle of such a

large project.

>> Um cool. So do you have a sense of what

space would be the best pa space to

start touring? Um

>> uh I think the first the good space

would be like just our website actually.

>> All right. If you want to go ahead and

start sharing.

Okay. So,

>> I believe I turned sharing on, but let

me know if it's saying you can't

So, wait.

I'm sharing my entire screen. I believe

>> it is just coming up black for me.

>> It is just coming up black. Okay. So let

Okay. Share screen.

>> Okay.

>> Oh, there we go. Great.

>> Okay. There you Okay. So,

so I think the good a good place like to

begin is our website. So that's where uh

every information is uh where we present

uh NF core. So we are a community that

collaborate open like open source next

floor component and pipeline.

This is what we do. We help we have like

organization user and developers. We

have some publications. So that's good.

uh this is all of the rules that we have

like for pipeline what we want to do for

the pipeline and everything that we do

is community own and I think for for us

that's that's really important so yes I

did say that what was important for us

reproducibility

uh collaboration but also openness

definitely openness is one of the

biggest like important one of the big

important thing for

uh we list on our website our events. Uh

we have some weekly uh or no bi-weekly

talks that we called by size where we

present like anything that could happen

in the community whether it's a new

pipeline that has been released or

someone wants to present something or we

present some new stuff in the community.

Uh I we have a yes community we have a

blog post we have a blog that we can

share like some news about about it.

This is all the events that we have at

the moment. So the coming the next

coming event that we will have will be

uh there are some trainings that are

being organized but I think these are

the trainings that are organized in the

community but because some of us are

like part of NF core and next community

like it's all tied in together um

also one last stuff that is interest oh

yes uh we have some special interest

groups that are like that we help and

that we host at NFC so that we can

uh group of community members that are

interested into working on a particular

aspect of NFore to work together. Uh and

I think especially the one that was the

most that really led to that was the

group working on animal genomics.

>> Cool. How many of these special interest

groups are there? It looks like there's

a couple of domain ones of

>> Yeah, really domain specific. So that we

have one on cancer, we have one for core

facilities, we have one dedicated to

education, imunology,

we try to make we uh basically we have a

process for that. So I can just go to GitHub.

GitHub.

So our community is quite big. We have

like uh more than 1, like we have almost

like 1,600 people on

on on our databation.

We have a bit less of like 200 repository

repository

and one of the latest repositories that

proposal proposals where we can where

like anyone in the community can make

like new proposal whether it's a new

pipeline a new special group or a new

request for command. uh and that way uh

that's the way that we can do that we

can use to improve uh whatever has been

done by the community uh what is done in

the community by the community

>> how many um

I'm curious there's it seems like there

was approximately a dozen special

interest groups I'm curious about how

many pipelines you have and >> oh

>> oh >> RFC

>> RFC

>> so pipeline at the moment Uh so we have

uh in total so pipelines that have been

released it's 84 pipelines that are

under development that's 43 and we have

12 pipelines that have been archived.

Usually when a pipeline has been

archived it means that it's either it

has it's it has been merged into another

pipeline or the technology is no longer

available and then it's no longer

anymore or anything.

>> Yeah. Not edited anymore but we keep the

code there because of like reproducibility.

reproducibility.

And at first we were really working towards

bionformatic and genomics but we keep

branching into like other field uh such

as proteomics which is still like pretty

much related to bionformatics. Uh we are

doing a lot of imaging. Uh we started

branching in in other field of science

such as satellite imagery and even economics.

economics.

>> Very cool. Oh yeah, I noticed you had a

neuroiming special interest group which

is um I actually learned to code back in

gosh 2008 2009 um because I was working

in a neuroiming lab and yeah to learn

>> sorry we have a pipeline that is

currently being worked on about like

cell paintings that really help us like

look at imagery uh data. So that one was

quite interesting.

>> Neat. Yeah, it's been it's been like a

good 15 years since I've done any

neuroiming. But um

>> yeah. Uh, I mean, part of me, which is

maybe it would be fun to go find some of

my old colleagues and be like, I just

want to ride along for a couple days

just to just to see where things are at

because I know there's just been huge

changes in

um the kinds of open source projects

that exist as well as the general like

um tool set available in um scientific

fields in general as well as neuro

imaging specifically between something

like 2010 and 2025. just a tremendous

amount of change. Um

>> Oh yeah. No, no, the fees are like

evolving like quite a lot and quite

fast. So yes, not that impressive. >> Um

>> Um

where are your um your RFC's?

>> Oh yes, sorry. So RFC's so RFC's at the

moment issues

uh if I look at label

Okay, at the moment we have uh 13 RFCs

that are like open.

uh I created like a few

because I'm quite interested I quite u

involved into this community and I

really like coding and doing stuff but

uh other of my colleague are involved

into that and other just member of the

of the community have been like involved

into I've been involved and and I've

been involved into that. I think the

latest uh RFC that was um merged in that

we actually used was AD advisory that uh

basically just catalog and list all of

the regression incompet and security

announcement about what was wrong and

what is no longer working in some of the

pipelines that we have.

>> This is great. I feel like um sometimes

it can be really hard to figure out if

there's a problem with an upstream

library where exactly the issue is. So

having like with like this dedicated area

area um

>> yeah no definitely that that was super

useful and uh it was proposed like uh it

was proposed not that long ago because

we started like all of that like

recently. Uh yes it was proposed in June

like this year because I think uh we

just started working on a ref in May. So

it was proposed in June and it was

accepted like quite fast because we and

then we started like putting some

information there.

>> I mean sometimes people just come up

with a good idea and everyone's like

yeah that's a good idea we should do it immediately.

immediately.

>> Yeah but then sometime it takes it takes

more time.

>> Yeah that's true. And it's that's not

necessarily because it's a bad idea.

Sometimes you just need to take longer

discussing something. I'm curious about

can we go back to the proposals? There

was one I was particularly intrigued by.

>> I think it was an open one.

>> Uh yes, code code owner for pipelines.

Can you tell me a little bit more about

that one?

>> Uh yeah, so that's one that actually is

uh so all of our all of our code for

pipeline is on GitHub. It's on NF core.

So let me if I just go back to join NF

core. So we have slacks that we use

daily. Uh GitHub this is where all of

our code is hosted. We communicate with

uh with everyone on Master Blue Sky,

LinkedIn and often on YouTube as well

and often when we have some meeting

together we used to use Gazertton but we

are switching toward like something else

at the moment and uh I I I don't think

we have anything uh I think we are

working with work adventure at the

moment but nothing uh like I don't think

we have taken any decision on that side

yet. So speaking of the pipeline, one of

the pipelines that I'm evolved with is

Sarak. So one thing important about

NFore, all of the pipeline should have a

completely boring name that really

describe what the pipeline is about. So

let's say another pipeline that I'm

involved in which is called RNA seek is

doing some RNA sequencing. So arac

that's very simple that's very boring

that's exactly what we want. We add a

couple of exception to that rule which

is one is sak which is a pipeline

uh that is that has to do with

sequencing as well but from world genome

sequencing or targeted sequencing and uh

that name existed before NFore. So it

had a community before NFore. So it's

one of the it's one of the few pipelines

that uh that have this uh this name that

is a nondescriptive name. And what we

decided with code owner

we look at some code

is that we decided that we uh so that's

me and Rick the other main developer of

the of the pipeline to be uh responsible

and to make sure that we can check

wherever a new PR has been made to our

repo. And this RFC is just saying, okay,

we just want to allow to make people

aware that this is possible and that we

should be doing that for the pipeline

that wants to be doing that. I don't

think this RFC is really necessary

because like some pipeline are already

doing that already and it was just for

me just a ways to formalize that idea.

>> Gotcha. So, is there um I guess I'm

curious about

what types of problems having a

designated conoder solves. So, for

example, something that I could imagine

being the kind of problem this would

help with is if you have, you know, 80,

what was it like 80 pipelines that are

active and like 40 more that are in

progress that sometimes people would not be

that's got to be a lot of different like

sort of lead maintainers of those

pipelines and if something needs

updating and that person is not

available, maybe there's not another

person who can do it. So like it might

make sense to have code owners from a

reliability perspective and maybe

possibly like require a certain amount

of responsiveness or activity from them.

Um but that's a little bit me projecting

onto you and the project based on my

experiences. So I'm curious about whether

whether

that's something that you thought at all

with regard to this or if there are

other sort of goals that

>> ownership would help. For for us, I

think like this code owner was more in a

way of safekeeping and making sure that

people that are responsible for a

pipeline can like safeguard the pipeline

toward what people could from the

community could put in the pipeline and

make sure that it adders to whatever the

main developer of the pipeline is

actually thinking in the way of the

pipeline. It's more for me it's more a

way to control and make sure that

nothing too out of like too

extraordinary is happening into a

pipeline. That should not happen without

the main developer and the maintainer of

that pipeline being aware of and of

course it definitely like owned by the

community. So whatever people want to do

in the pipeline is completely open and

we usually work with each other quite

often and I like even I I often

make some small pull request to other

pipelines that I'm completely not

responsible for just to help other

people. So that's something that can

happen. But in such a case uh because

I'm both a maintainer and an

administrator for the for the community

I don't need like such a such a good

owner like ownership because yes I don't

have this I have like specific powers

that have been like

imparted onto me. uh but definitely for

other people that's something that could

be interested and I think that's why we

wanted like to to do that to make sure

that uh okay if you do that then I want

to make sure that I can see the pull

request happening and that why I can ask

okay you've done that that is good but

maybe we need to add that we need to add

some test we need to make sure that

everything works well let's update the

documentation as well and so on

>> for um so is each pipeline a GitHub

repository within in the organization.

>> Exactly. Uh so all of the repositories

that we have them most of them are for

pipeline. So s this is my this is the

pipeline that I've been working on. Uh

MC micro that's another pipeline. That's

like other pipeline. This is our

website. This is all of the

configuration that can be used at

multiple institution.

That's really something that is specific

to Nexflow and not specific to NF

actually people outside of NFore can use

this configuration to

parameterize their like and to control

their own

their own next flow pipeline

uh module that will be like some

specific components that have that are

the building block of follow pipeline

sorry module

>> no worries

>> let's say Like here I'm looking at a

module. So a module for me and I will

look at one of the ones that I know most.

A module will be just the smallest unit

of block uh that we can do and uh that

just execute common line.

>> Gotcha. And it looked like from looking

at the um repositories that they get

updated quite frequently. Like all of

the ones that you were looking at were

which were at the top. So the most

recently updated, but they were all

updated today. Um

>> Oh yeah. No,

>> this this uh this repository is the one

that is updated like the most often and

is the repository that is used uh all

over the place by everyone.

uh because the way that we did stuff at

NFore is that you don't need to have an

NFore pipeline to use NF core. NF core

is at the same time uh the pipeline that

we that we build the community but also

all of the components that we have built

around Nexflow and around Nore and all

of the tooling that we help develop. Uh

so for me the module will be just the

building bricks the building bricks that

we can use to build our pipeline and we

have modules which should be like the

small units sub workflow which will be

like an assembly of multiple module and

that we can use to import in our pipeline.

pipeline.

>> Gotcha. So the um can you go back to uh

the list of repositories? >> Yes.

>> Yes.

>> Gotcha. So some of these other ones are

like specific like differential

abundance is a pipeline I'm guessing.

>> Yeah, differential abundance is a

pipeline uh tools that will be like the

Python package that we build with helper

tools for the Nefar community and with

tools came like one of the most center

stuff for community which is basically

the what we call the template which is a

skeleton that we use for all of our

pipeline. That way we know that our

pipeline have a more or less a common

structure and are easy to review even if

you don't know the underlying u

like science or like the underlying like

logic behind the pipeline you if they

all follow the same structure it's

easier to review.

>> Can we go into one of the specific

pipelines? Could be the differential

abundance one could be something that's

just like a just an average pipeline. Um

how many so it looks like a lot of the

files haven't been touched in a while

but it did get updated today.

>> Yeah. So a lot of file uh so this

pipeline so this we are looking at the

master branch. So that will be the

latest release. If you want to look at

what is happening in development we need

to check the dev branch and we can see

that some stuff happened yesterday.

>> Gotcha. Gotcha. um is the Yeah.

No. So it's a it's a this is a pretty

this is a like I would say it's not one

of the it's one not one of it's I think

it's one of our like top pip I think we

can sort it by yes we can sort in term

of popularity so the I think pipeline

would be like the most

famous pipeline s would be the second

one and then like uh I think

differential abundance should be not

Does each pipeline have its own sort of

release process? Is it standardized

across the p pipelines? Um

>> yes, the release process is standardized

across pipeline.

>> Gotcha. And is do you have sort of is it

sort of like delegated like

um there's someone who is a maintainer

on the GitHub repository that has the

ability to like click the merge button

on the pull requests or how does that

get structured?

Uh so basically any uh the way that we

work at the moment and that's that's one

of the stuff that we've been talking uh

when we were meeting like the the the

the the

administrator the core team as we say

uh is uh so basically the way that we do

a release uh let's let me just check

with the pipeline that is currently in

the process of being released. Uh so we

have a PR that is doing the release that

uh merge all of the commit from dev to master

master

and as we can see on this particular

pipeline. So this pipeline is quite a

beast quite a monster. The latest

release was eight months ago

and this is the new release that is

coming on and we can see that there are

>> Gotcha. And do you know have a sense of

how how long there is between releases?

>> Uh it really depends from like one

pipeline to the other. Uh I think the

way that we

I think it really depends on the on the

people that are developing the pipeline.

If you manage to have like a team of

people that is working with you or if

you're all alone on your pipeline and it

depends also on the size of the

pipeline. Some pipeline can be released

like more often also depending if

if there are like more bug fixes or if

like all of the underlying tools have

been updated or some of the API from the

tool that we are using gets changed or

something like that. like a lot of the

pipelines that we do are dependent on

the tools that other people have been

developing. So we are collaborating

quite a lot with other communities as well.

well.

>> For the pipelines do they have um

well you have these other repositories

like modules and tools are those things

that are getting used within the pipelines?

pipelines?

>> Uh yes definitely. So the way that we so

let me let's go back to uh let's go back

that was our example. So I showcased

before the module repo and if I look at

here in module

here all of this uh all of this u

all of these folder are modules that

have been like imported uh from the

module repo. Are they like um like live

imports or symbolic links or did you

like did they get copied at some point?

>> Uh they have been copied they have been

copied. So the way that we work with

that we have a JSON file that lists for

every module where is coming from. So we

know that this module are coming from an

Fore because you could have your own

fork of of the module repo. Let's say

you're a private company that wants to

do some specific stuff that you don't

want to be put in into like the

like into the wild. So you can have your

own your own stuff made there or you're

just working on some experimental stuff

and you don't want it to be like public

yet or like you could do whatever you

want. And basically for each of these

module we have the g commit that

correspond to the latest time that it

was updated.

So basically we re we re reimplemented

uh an installer to grab like all of the

code from the module repo into this

other repo. I'm curious about

>> We decided Yeah. Sorry.

>> No, go ahead. Finish finish your thought.

thought.

>> No, we decided to do that because we

figured it would be more efficient for

us to have actual uh real code into our

pipeline that to have like a sim link or

g module. We noticed that sometime it

might not have been like as efficient as

we wanted.

>> Makes sense. when did you sort of um

come up with this standardized modules.json

modules.json

syntax and like this feels like it was a

solution to something uh and wouldn't

have been there from the very beginning

and I'm curious about like where it

arose and like how you decided to go

this route.

>> No, definitely it wasn't there at the

beginning. So at the beginning next flow

had like another syntax which we call DSL1

DSL1

and at that time uh we had no shared

module. It's when we came up with it's

when next floor came up with the new

version of the syntax which is DSL2 that

we came with the model repo and we came

up with that by a lot of discussion into

figuring out like okay how do we want to

name things how do we want to put them

what should be a module how should a

module be composed and how should we how

do we want to organize everything and uh

yes that all led to the creation of of

this repo um that like host all of the

file for nextl. >> I'm

>> I'm

and I think yeah and I think this

started in 2020 sorry

>> 2020. Um I'm guessing from the naming

schema that DSL 2 and DSL1

uh were not compatible and that like

switching from one to another like

required um whole migration progress.

Yes. Uh the migration was uh was a bit

hard. Uh I think some of the old

pipeline might be still held back in

DSL1. So the main issue with that is

that if your pipeline is still stuck in

DSL1, you cannot use the latest version

of Nexflow. You have to use like a

previous version of Nexflow. But that's

not really a problem because it's easy

to install anyway and uh it's easy to

just use and like uh switch version.

I feel like like major migrations are

one of the biggest challenges that

communities can face and they can take

forever. So I'm curious about your

thoughts about the

uh migration

um that you guys went through and the

sort of process you undertook. If you

have any sort of like like best

practices or lessons you learned from it

that you think um

>> uh yes so we under we we went through

like a lot of different migrations. I

think the first one was DSL1 to DSL 2.

Uh that was a big one and it took us

like some time. I think for Sak uh it

took us like quite a while.

I don't I don't really remember like how

long it took us but it took us like

several months. Uh because nothing was

like you had to migrate like everything

at once and that was the issue. uh some

other migration were uh definitely

easier but it uh it was uh it was

complex to

put into full usage. So let's say for

the module repo what we do we test every

module separately as it's on unit uh we

are very dedicated to our testing and we

do try to test everything that we make

sure that everything works and that we

don't risk to install stuff that is not

working in the pipeline. So every PR

that we make on the model repo uh needs

to be successful and need to make sure

that it works well. So if I have a look

at this PR, I have no idea what happened

there, but I can look at the PR, I can

look, okay, it handle like a specific

input for a specific tool

and I can see that here some test were

not working. I can see that some people

made some review comment on how to

improve stuff and this is a long p

actually. Sorry,

>> you ignore it.

>> I could see that I could see that in the

end uh there was some code that was

updated and here we have all of our

tests that went through. Here we had

like 40 58 different test and uh they

all went through and one of the major

stuff that we started working I think it was

was

two years ago maybe uh was migrating all

of our test uh infrastructure from pyest

uh which is like a testing framework in

Python uh that we basically re revert

that we use to help us like work with

next flow. it was a bit of a h so it was

not that helpful and then uh some people

in the community developed NF test and

we've been using that

ever since and that's dedicated

framework that is uh wrapped around next

and that help us to work with that help

us write our test and we did this world

migration over the last like two was it

two years or was it more I don't

remember about that but yes we under go

that world migration and we just

finished that like a couple of months ago.

ago.

>> I'm curious about this might be a bit of

a digression but whatever there are no

rules. Um, I'm curious about the NF test

like what what changes or customizations

you made that uh piest didn't have or

that like what is it like um

yeah I'm just curious about how you

customize your tests.

>> Uh yeah so basically uh so what we do at

the module level we do we really do

unique test uh unit test. So we we want

to make sure that the So let me uh maybe

I can find like an old pull request that

Uh yes, let's look at this one for example.

example.

So the way we were doing so okay so

that's the script itself. So we don't

really care that's the NF test. So we

are we be looking into that later.

Uh okay. So that's the way that we are

writing our test before. So basically

that's the A that we found uh to write

to wrap our module as uni as a single

unit and to allow us to execute that we

were executing all that within within

pyest and basically that was the command

that we were using and from that command

this is the output that we were checking.

So that was how we were doing stuff

before in Pyest. The way that we do

stuff now with NS test basically we

write the test directly uh within the

new framework

uh and that's the way that we write test

uh here. Yes.

Right. Yes. So we import the script that

is just uh

one one layer up one folder up. We want

to run the this particular process. Here

we give some specific uh argument to the

pipeline on it which should be working

or not. And here we want to capture all

of this information and uh print that

into a snapshot. And here if I look at

the snapshot I can see some in MD5

content for some specific file.

And that's really what we want what we

want to do. And that way is a bit is

definitely more easier for us because we

don't hardcode like any particular MD5

here in the file. So it's much easier

for us to update compared what we were

doing before where we had to run this

once check what was the MD5 sum and put

it in the test.

>> Gotcha. Gotcha. Um switching gears back

to the RFC's maybe. Um, I'm curious

about talking about >> a

>> a

what one of the most like either

controversial or just like most

participated in community decisions

you've made is. So maybe that's like the

RFC with the most comments, but maybe

it's not actually an RFC at all. Or

maybe it's got fewer comments, but

they're really intense comments. I'm

just curious about like a decision that

you had to make that's been something

where it wasn't quick and easy. Um, so

it doesn't necessarily have to have been

like full of drama, but like something

that people really had to talk things through.

through.

>> Yeah. No, I think if I if I'm looking at

something that could be like uh contro

like a bit more controversial, it would

be more in

Yeah. Uh definitely uh I think for me it

would be more pipeline proposal.

Pipeline proposal. So this one was accepted

accepted

and uh the discussion went on into like uh

uh is there an overlap with other pipeline?

pipeline?

Uh if there was an overlap like what is

the overlap?

Can we do like

can we contribute to that? Then there

was some more question about what the

pipeline is actually doing. how should

we uh organize the pipeline and so on.

And we decided in the end that okay

there are like some stuff that are like

pretty similar to other pipeline but

it's different enough that it's uh it

should be like its own pipeline and yes

so for us like the discussion really

went on uh with multiple people from the

community uh that went on and like

discuss why this one should really be

into into NF core. And what we really

like at NF Core is for Pipeline, we like

to make a a nice subway map. >> Oh, it's so cute.

>> Oh, it's so cute. Um, so what was the was the end result

Um, so what was the was the end result that it got created as a separate

that it got created as a separate pipeline?

pipeline? >> Uh, yes. So the end result was this one

>> Uh, yes. So the end result was this one was accepted as its own pipeline.

was accepted as its own pipeline. Uh, because definitely I can see label

Uh, because definitely I can see label accepted. So this one was accepted. Uh I

accepted. So this one was accepted. Uh I think we just started to rule out like a

think we just started to rule out like a system uh I think it's uh I think yes we

system uh I think it's uh I think yes we just started to rule out like a system

just started to rule out like a system where we can uh approve uh

where we can uh approve uh things. Oh yes and no now we are we are

things. Oh yes and no now we are we are waiting for like a particular corum to

waiting for like a particular corum to approve or not and I can see that the

approve or not and I can see that the bot was not working on this particular

bot was not working on this particular PR. So yes some stuff that we need to

PR. So yes some stuff that we need to fix too. Yeah. Are you is that like a

fix too. Yeah. Are you is that like a custom bot you created? Uh

custom bot you created? Uh >> yeah, we created a custom bot. That's

>> yeah, we created a custom bot. That's one of the uh multiple like other repos

one of the uh multiple like other repos that we have here in NFore. Uh a lot of

that we have here in NFore. Uh a lot of the repo are pipeline. Some of the repo

the repo are pipeline. Some of the repo are help to that help us with the

are help to that help us with the pipeline but also some of the other pip

pipeline but also some of the other pip some of the other

some of the other things. Uh

things. Uh >> you had like a one.

>> you had like a one. Sorry,

Sorry, >> you also had like a website repo and

>> you also had like a website repo and like a yeah

like a yeah pipelines but some are other things that

pipelines but some are other things that are not pipelines.

are not pipelines. >> We have GitHub actions that help us like

>> We have GitHub actions that help us like set up our test. We have specific

set up our test. We have specific scripts for specific scripts for

scripts for specific scripts for infrastructure. Oh yes, I forgot the

infrastructure. Oh yes, I forgot the infrastructure team as well that is

infrastructure team as well that is really dedicated to to help us with all

really dedicated to to help us with all our infrastructure. And yes, another

our infrastructure. And yes, another team that I forgot was a team that we

team that I forgot was a team that we just created like recently that is

just created like recently that is dedicated toward documentation.

dedicated toward documentation. >> Cool. Um, and then what kind of thing

>> Cool. Um, and then what kind of thing what kind of things does the bot do in

what kind of things does the bot do in general?

general? >> Uh, so the bot help us. Uh, so

>> Uh, so the bot help us. Uh, so that's very interesting. Uh let's have a

that's very interesting. Uh let's have a look at yes I always take take this

look at yes I always take take this pipeline as an example because it's the

pipeline as an example because it's the one that I've been working like quite a

one that I've been working like quite a lot. Next time

>> right no sorry it will be but so basically the bot will help us into

so basically the bot will help us into making the new uh PR that are updating

making the new uh PR that are updating the skeleton the template

the skeleton the template and the b creates a PR and uh this is

and the b creates a PR and uh this is all automate auto automated and when the

all automate auto automated and when the PR has been like done. U I can have a

PR has been like done. U I can have a look into it checking what the merge

look into it checking what the merge conflict are and fixing all of the stuff

conflict are and fixing all of the stuff and then continue working until

and then continue working until everything works well and sometime it

everything works well and sometime it take a while. Yeah.

take a while. Yeah. >> Um

>> Um >> that's Yeah.

>> that's Yeah. >> Can you scroll up to the sort of

>> Can you scroll up to the sort of original post? See what the

original post? See what the >> original?

>> original? Yeah. This one is a bit uh there should

Yeah. This one is a bit uh there should be like just the one but yes. So the bot

be like just the one but yes. So the bot the bot has done like all of this commit

the bot has done like all of this commit but uh we've been using this pipeline as

but uh we've been using this pipeline as a as a testing repo. So maybe another

a as a testing repo. So maybe another pipeline will be like if we check again

pipeline will be like if we check again at the differential moment that will be

at the differential moment that will be easier to check

easier to check the but has been doing.

>> So it open this one >> open this one. There is just one P that

>> open this one. There is just one P that is open. Uh I can see that there is some

is open. Uh I can see that there is some conflict that hasn't been resolved yet.

conflict that hasn't been resolved yet. >> And the template is that the um PR

>> And the template is that the um PR templates or is it like an NF core

templates or is it like an NF core template?

template? Um

Um >> uh so it's it's

>> uh so it's it's >> what I what I call template will be like

>> what I what I call template will be like the skeleton that we use for our

the skeleton that we use for our pipeline and uh that's something that we

pipeline and uh that's something that we try to uh to keep up to date with

try to uh to keep up to date with coming with the update that are coming

coming with the update that are coming to the next flow syntax. And basically

to the next flow syntax. And basically we are uh

we are uh we are six months after like every next

we are six months after like every next flow every next flow release and every

flow every next flow release and every stuff that is coming in next flow uh and

stuff that is coming in next flow uh and that are mature we incorporate them into

that are mature we incorporate them into ancore and usually we uh we lag like six

ancore and usually we uh we lag like six months behind.

months behind. >> Gotcha. Okay. That's part of the we have

>> Gotcha. Okay. That's part of the we have part of some of the RFC we have at the

part of some of the RFC we have at the moment are dedicated actually to that.

moment are dedicated actually to that. So when there's a a syntax update to

So when there's a a syntax update to um the the templates

um the the templates that impacts all the pipelines and so

that impacts all the pipelines and so once that decision's been made through

once that decision's been made through whatever process I don't know whether

whatever process I don't know whether it's always an RFC or whether it's

it's always an RFC or whether it's something that maybe one of the uh

something that maybe one of the uh committees or teams that you talked

committees or teams that you talked about earlier does but like once that

about earlier does but like once that decision gets made then you can roll

decision gets made then you can roll that out to all of the pipelines through

that out to all of the pipelines through this bot that sort of like opens up a

this bot that sort of like opens up a pull request on every repo with the

pull request on every repo with the syntax changes.

syntax changes. >> Exactly. So basically the way we it

>> Exactly. So basically the way we it works is that every time so here this is

works is that every time so here this is the Python package that we help with a

the Python package that we help with a lot of helper tools that help us to uh

lot of helper tools that help us to uh to automate as much as we can because I

to automate as much as we can because I think like uh most of like most of us

think like uh most of like most of us come from like binformatics and I like

come from like binformatics and I like to say that we are lazy and we try to

to say that we are lazy and we try to automate stuff as much as possible.

automate stuff as much as possible. >> Yeah. I don't know if you have this,

>> Yeah. I don't know if you have this, probably not, but there's something in

probably not, but there's something in the US and I think maybe England too

the US and I think maybe England too called a lazy susan, which is like a

called a lazy susan, which is like a thing that um it's like a thing that

thing that um it's like a thing that rotates. So, you can put it on a table

rotates. So, you can put it on a table and you can put like uh salt and pepper

and you can put like uh salt and pepper or like mustard or ketchup or whatever

or like mustard or ketchup or whatever on it and then instead of someone having

on it and then instead of someone having to like pick something up and hand it to

to like pick something up and hand it to you, you can just rotate it. Anyway,

you, you can just rotate it. Anyway, it's called a lazy Susan, but I always

it's called a lazy Susan, but I always call them efficient Susanss because it's

call them efficient Susanss because it's just it's just being efficient.

just it's just being efficient. >> Yeah. No, no, no. I see. I like that. So

>> Yeah. No, no, no. I see. I like that. So basically here in this repro uh so we

basically here in this repro uh so we have a lot of Python code and then we

have a lot of Python code and then we have what we call or like a pipeline

have what we call or like a pipeline template that anyone can use to build a

template that anyone can use to build a pipeline whether it's a pipeline that is

pipeline whether it's a pipeline that is for course or for our usage here in the

for course or for our usage here in the community or maybe even for your own

community or maybe even for your own pipeline that is that exists outside of

pipeline that is that exists outside of the community.

the community. A lot of people are using the uh Nefar

A lot of people are using the uh Nefar template to build pipeline because it's

template to build pipeline because it's it's built in with like a lot of stuff

it's built in with like a lot of stuff and we build the template in such a way

and we build the template in such a way that it's highly modular and you can uh

that it's highly modular and you can uh set up a lot of stuff or you can disable

set up a lot of stuff or you can disable a lot of stuff. Uh I think it might be

a lot of stuff. Uh I think it might be tracing maybe uh

tracing maybe uh let me try to Yes. Okay.

Uh wait, I think the command is. So uh the team behind like

So uh the team behind like NF tools has built like a graph

NF tools has built like a graph wonderful like graphical interface in

wonderful like graphical interface in the terminal that help you like build

the terminal that help you like build the pipeline. And basically here if you

the pipeline. And basically here if you create your pipeline you can say okay

create your pipeline you can say okay it's a pipeline that is for NF core or

it's a pipeline that is for NF core or it's a pipeline that is custom and it's

it's a pipeline that is custom and it's if it's custom you type in like whatever

if it's custom you type in like whatever you want for your pipeline you write the

you want for your pipeline you write the description you write the name and then

description you write the name and then you can decide which feature you want to

you can decide which feature you want to use or which feature you don't want to

use or which feature you don't want to use. So should it be like something that

use. So should it be like something that is on GitHub or not? Does it has a

is on GitHub or not? Does it has a badge? Do you want a change log license?

badge? Do you want a change log license? Do you want an MIT license file? Do you

Do you want an MIT license file? Do you want test and everything and so on. So

want test and everything and so on. So we kind of like we build the the wall

we kind of like we build the the wall stuff around that so that the wall

stuff around that so that the wall template is fairly um modular and easy

template is fairly um modular and easy to easy to use for any newcomers.

to easy to use for any newcomers. >> Yeah, this looks incredible. I also

>> Yeah, this looks incredible. I also really appreciate that like on the very

really appreciate that like on the very first page it said something along the

first page it said something along the lines of if you want this pipeline to be

lines of if you want this pipeline to be an NF core like please talk to us first.

an NF core like please talk to us first. Yeah, that's that's one of the most

Yeah, that's that's one of the most issue that we have. Uh I think I think

issue that we have. Uh I think I think that's one of our main issue is that

that's one of our main issue is that often people come with a pipeline that

often people come with a pipeline that is like fully complete and

is like fully complete and we are we are like a community that is

we are we are like a community that is like collaboration first and for me if

like collaboration first and for me if someone comes with a pipeline that is

someone comes with a pipeline that is fully complete maybe something that we

fully complete maybe something that we missed and that it should have been like

missed and that it should have been like a collaboration at the very beginning.

a collaboration at the very beginning. So that's something that we try to work

So that's something that we try to work on and to improve. Uh so yeah, that's

on and to improve. Uh so yeah, that's why it's listed like everywhere because

why it's listed like everywhere because we want to make sure that okay, people

we want to make sure that okay, people collaborate before they do anything.

collaborate before they do anything. >> Yeah, I mean I think this is a really

>> Yeah, I mean I think this is a really important and I don't know if there's

important and I don't know if there's any completely universal best practice,

any completely universal best practice, but a near universal best practice. I

but a near universal best practice. I feel like there's very few things where

feel like there's very few things where you don't benefit from talking to the

you don't benefit from talking to the community and the maintainers ahead of

community and the maintainers ahead of time because having to go back and make

time because having to go back and make huge changes that could have easily been

huge changes that could have easily been prevented because by just having a

prevented because by just having a conversation and being in touch with

conversation and being in touch with them from the beginning.

them from the beginning. And

And >> and also also I think like one of the

>> and also also I think like one of the big force of like making a pipeline

big force of like making a pipeline proposal like very early is that you

proposal like very early is that you could find like other people that are

could find like other people that are interested into your into the pipeline

interested into your into the pipeline and gaining like early on some

and gaining like early on some other collaborator or other contributors

other collaborator or other contributors towards that pipeline and that will

towards that pipeline and that will really be helpful towards the quality of

really be helpful towards the quality of the code actually.

the code actually. >> Yeah. All right. We're sort of getting I

>> Yeah. All right. We're sort of getting I booked off 90 minutes for this and also

booked off 90 minutes for this and also I feel like there's only so long anyone

I feel like there's only so long anyone could possibly want to watch it. So I

could possibly want to watch it. So I feel like we should start wrapping up

feel like we should start wrapping up even though there's a thousand

even though there's a thousand additional questions I have for you. I

additional questions I have for you. I wonder is there an area or a topic that

wonder is there an area or a topic that you think um we should go over before

you think um we should go over before wrapping up.

wrapping up. >> Yeah. One last thing that I wanted to

>> Yeah. One last thing that I wanted to say was like part of the documentation.

say was like part of the documentation. So we have some documentation that is uh

So we have some documentation that is uh quite central about what we should do

quite central about what we should do like in the community some contribution

like in the community some contribution um

um we have contribution about like NF test

we have contribution about like NF test about like all of our all of our

about like all of our all of our component uh we give out some advice

component uh we give out some advice about which like how we code how people

about which like how we code how people should be contributing we have some

should be contributing we have some guideline we have some tutorial we

guideline we have some tutorial we collaborate quite often as well with the

collaborate quite often as well with the next community and with people at Saka.

next community and with people at Saka. So we have training that has been like

So we have training that has been like uh made and that is like amazing.

uh made and that is like amazing. Definitely we recommend like this

Definitely we recommend like this training for every newcomer

training for every newcomer like in next row

like in next row uh quickly. Oh yes and what I wanted to

uh quickly. Oh yes and what I wanted to say uh we have documentation we have

say uh we have documentation we have documentation about the graphic design

documentation about the graphic design and especially for the

and especially for the workflow uh diagram and for and for the

workflow uh diagram and for and for the subway map

that was a workflow diagram. Yes, we have some subway map as well here. And

have some subway map as well here. And yes, basically that's all of the stuff

yes, basically that's all of the stuff that we have done is there. And I think

that we have done is there. And I think yes, one of the ones that I'm the most

yes, one of the ones that I'm the most proud of would be like uh this subway

proud of would be like uh this subway map that is animated.

map that is animated. >> Oh, wow. Oh, that's incredible. I love

>> Oh, wow. Oh, that's incredible. I love it.

it. >> Yes. It's made in pure SVG, so it works

>> Yes. It's made in pure SVG, so it works without any problem. Anyway,

without any problem. Anyway, >> incredible. Incredible. Sorry, I'm now

>> incredible. Incredible. Sorry, I'm now I'm very distracted by this animation

I'm very distracted by this animation which I feel like I could watch.

which I feel like I could watch. >> Yeah. Sorry. Sorry.

>> Yeah. Sorry. Sorry. >> No, it's great. I might just come back

>> No, it's great. I might just come back to it later and just like stare at it

to it later and just like stare at it and meditate.

and meditate. >> Yeah. Oh, yes. I talk about the this is

>> Yeah. Oh, yes. I talk about the this is how we are organized as a maintenance as

how we are organized as a maintenance as a the wall or the wall community

a the wall or the wall community and I think yes we have some statistic

and I think yes we have some statistic as well. Obviously

as well. Obviously Phil loves statistics. So that's why we

Phil loves statistics. So that's why we have like a lot of statistic and seeing

have like a lot of statistic and seeing the world like community in numbers in

the world like community in numbers in term of like

term of like Slack, GitHub, Twitter or whatever.

Slack, GitHub, Twitter or whatever. I think the

I think the pipeline numbers. Oh yes. So we can see

pipeline numbers. Oh yes. So we can see that we have a lot of pipelines that

that we have a lot of pipelines that have been like released or in

have been like released or in development and I think we are like

development and I think we are like almost getting like towards plateau. I

almost getting like towards plateau. I think maybe I don't know. I think only

think maybe I don't know. I think only time will tell.

time will tell. Um, I had a couple more questions. Um,

Um, I had a couple more questions. Um, for your, it looks like you have a bunch

for your, it looks like you have a bunch of different ways for people to start

of different ways for people to start contributing. Do you feel like there are

contributing. Do you feel like there are some common like new contributor

some common like new contributor pathways that are

pathways that are um particularly successful or like

um particularly successful or like common common pathways for people to

common common pathways for people to take?

take? Uh I will say usually like the first way

Uh I will say usually like the first way that people communicate with us will be

that people communicate with us will be like either on uh on GitHub by creating

like either on uh on GitHub by creating an issue when something is not working

an issue when something is not working or or coming like towards us on Slack.

or or coming like towards us on Slack. Oh, I didn't show Slack as well. Uh yes.

Oh, I didn't show Slack as well. Uh yes. So this is Slack. So uh I'm on every

So this is Slack. So uh I'm on every possible Slack channel that we have in

possible Slack channel that we have in the in this in this workspace. So I have

the in this in this workspace. So I have a lot of stuff going around. Uh

a lot of stuff going around. Uh basically we have a dedicated slack

basically we have a dedicated slack channel for every pipeline and for some

channel for every pipeline and for some pipeline we have a dedicated uh

pipeline we have a dedicated uh dev channel so that the people that are

dev channel so that the people that are actively developing that pipeline can

actively developing that pipeline can work together and then uh I have some

work together and then uh I have some specific like u private group because I

specific like u private group because I like uh creating channel and then I

like uh creating channel and then I created like many many uh other channel

created like many many uh other channel uh

uh oh yes we created a channel to just

oh yes we created a channel to just request review because that's what like

request review because that's what like we're a community we're community first.

we're a community we're community first. So if you need someone to review your

So if you need someone to review your code for an F of code you can ask people

code for an F of code you can ask people to review your code there. uh we have

to review your code there. uh we have channel where we can just be angry at

channel where we can just be angry at stuff and like just rent and saying okay

stuff and like just rent and saying okay this stuff is not working today and I

this stuff is not working today and I think the

yes we have a channel where we just thank people saying okay this people is

thank people saying okay this people is an amazing person and they've been

an amazing person and they've been they've been really really helpful so

they've been really really helpful so yes we we've been like

yes we we've been like >> yes I think we are quite quite a good

>> yes I think we are quite quite a good community where uh like I haven't seen

community where uh like I haven't seen like much issues like like I'm not aware

like much issues like like I'm not aware of any issue in the community. I know

of any issue in the community. I know that we have a safety group. I know that

that we have a safety group. I know that they have replied to some stuff and they

they have replied to some stuff and they have worked stuff but I have no idea

have worked stuff but I have no idea what happened because it wasn't that at

what happened because it wasn't that at least like from what from my own

least like from what from my own understanding it wasn't that big of a

understanding it wasn't that big of a deal that us the core member had to

deal that us the core member had to intervene towards it.

intervene towards it. >> Yeah. Yeah.

>> Yeah. Yeah. >> So, I think we're a pretty benevolent

>> So, I think we're a pretty benevolent community and I'm quite uh happy about

community and I'm quite uh happy about that.

that. >> Yeah. And it's also true that like um

>> Yeah. And it's also true that like um part of what enables people to be happy

part of what enables people to be happy in a community is knowing that there are

in a community is knowing that there are spaces for conflicts to get worked out

spaces for conflicts to get worked out and for issues to get handled. Um so

and for issues to get handled. Um so often having like a space where people

often having like a space where people feel happy and celebrate each other and

feel happy and celebrate each other and are kind to each other goes handinhand

are kind to each other goes handinhand with also having spaces where there are

with also having spaces where there are issues. Um because anything of

issues. Um because anything of significant size and this is definitely

significant size and this is definitely a significant size is going to have some

a significant size is going to have some kinds of conflicts or issues. So um yeah

kinds of conflicts or issues. So um yeah >> uh no addition

>> um a couple of other quick questions. Um do you so you have the maintainers of

do you so you have the maintainers of pipelines. Um, I'm curious in general,

pipelines. Um, I'm curious in general, often projects will have problems with

often projects will have problems with getting people who can step into those

getting people who can step into those kind of maintainer or leadership roles.

kind of maintainer or leadership roles. I'm curious if that's an issue you've

I'm curious if that's an issue you've ever had or if you have like any sort of

ever had or if you have like any sort of systems, processes, cultural things in

systems, processes, cultural things in place to sort of help folks go from

place to sort of help folks go from someone who's just joining the community

someone who's just joining the community uh to someone who's taking more of a

uh to someone who's taking more of a leadership role.

leadership role. Uh I don't think we like basically the

Uh I don't think we like basically the main stuff the main way that we work

main stuff the main way that we work with that will be uh

with that will be uh uh pipeline really depend on the people

uh pipeline really depend on the people that work on their pipeline and guess

that work on their pipeline and guess I'm just showing this uh recent article

I'm just showing this uh recent article uh because it showed

where was crash it was in the prep

it was in the prep Uh

Uh okay. So very

Okay. uh because anyway what I wanted to say was uh the main issue I think for

say was uh the main issue I think for the pipeline is that often we have like

the pipeline is that often we have like people that are really dedicated towards

people that are really dedicated towards the pipeline because they're doing

the pipeline because they're doing either PhD a postoc or it's a pipeline

either PhD a postoc or it's a pipeline that is being useful for a specific

that is being useful for a specific infrastructure so if you find

infrastructure so if you find collaborator into that then multiple

collaborator into that then multiple people can work onto that for some

people can work onto that for some specific uh

specific uh pipeline you can see that uh the

pipeline you can see that uh the contributor have been like shifted over

contributor have been like shifted over from people to people. I think we can

from people to people. I think we can see that pretty well with the M&

see that pretty well with the M& pipeline.

pipeline. If I look at the collaborator,

if I look No, not the collaborator inside. Sorry. Contributors. That was

inside. Sorry. Contributors. That was it. If we look at the contributors list,

it. If we look at the contributors list, we can see that uh Phil for example was

we can see that uh Phil for example was pretty active like very early on that we

pretty active like very early on that we can see that Arshield uh joined in like

can see that Arshield uh joined in like uh quite a lot and he was quite active

uh quite a lot and he was quite active in in 2020 until like 2022. Then we can

in in 2020 until like 2022. Then we can see that John here join in like later on

see that John here join in like later on and I join like later on as well and we

and I join like later on as well and we can see that a lot of the contribution

can see that a lot of the contribution can like shift from one pipeline to

can like shift from one pipeline to another. Uh but for me the pipeline

another. Uh but for me the pipeline belong to the community. So it's not

belong to the community. So it's not really like owned by a particular like

really like owned by a particular like person but it's more owned by a group of

person but it's more owned by a group of people. And what we try to do at our

people. And what we try to do at our level is just to make sure that uh

level is just to make sure that uh people can work together and we can help

people can work together and we can help them provide with a good framework where

them provide with a good framework where like if someone is no longer developing

like if someone is no longer developing a pipeline then people can come and take

a pipeline then people can come and take over it

over it >> right is do you have like a process for

>> right is do you have like a process for it or is it more just like on a case by

it or is it more just like on a case by case basis you

case basis you >> it's more a case to case it's definitely

>> it's more a case to case it's definitely a case to case basis

if I remember. Well, yes, for example, this one uh there was a proposal to

this one uh there was a proposal to create like a new pipeline, but

create like a new pipeline, but basically this new pipeline would

basically this new pipeline would replace a pipeline that was out of date.

replace a pipeline that was out of date. So we decided, okay, let's just use this

So we decided, okay, let's just use this new proposal for a pipeline to take over

new proposal for a pipeline to take over an older pipeline.

an older pipeline. >> Gotcha. So it like the you wouldn't like

>> Gotcha. So it like the you wouldn't like rename the pipeline. Instead, you'd sort

rename the pipeline. Instead, you'd sort of merge the changes and give the

of merge the changes and give the >> uh I'm I'm not sure whatever we decided

>> uh I'm I'm not sure whatever we decided like in the end. I think we we said that

like in the end. I think we we said that Oh, yes, definitely. I think we're going

Oh, yes, definitely. I think we're going to keep the name of the original

to keep the name of the original pipeline.

Uh wait stick with the same same M repo but just yeah so we will keep the we

but just yeah so we will keep the we will keep the name the original name of

will keep the name the original name of the pipeline which was KIC and not use

the pipeline which was KIC and not use case row because I think Kik is more

case row because I think Kik is more like boring

>> um in a case where a pipeline just got completely replaced is that something

completely replaced is that something that would cause breaking changes for

that would cause breaking changes for people who are using the pipeline or is

people who are using the pipeline or is there

there >> um like until like uh since like

>> um like until like uh since like everything is still uh like every

everything is still uh like every release that ever happened will still be

release that ever happened will still be listed on NF core. We never delete

listed on NF core. We never delete anything.

anything. >> We always keep everything everything

>> We always keep everything everything there. So everything that has been

there. So everything that has been working is still working and should

working is still working and should continue to work until like let's say

continue to work until like let's say like bioonda is broken or bioontainer is

like bioonda is broken or bioontainer is no longer working or all of the

no longer working or all of the containers that we're using are no

containers that we're using are no longer working anymore.

longer working anymore. >> Gotcha. And users tend to pin to a

>> Gotcha. And users tend to pin to a specific version of

specific version of >> definitely. Yes, that's that's the way

>> definitely. Yes, that's that's the way that we we recommend people to run a

that we we recommend people to run a pipeline. And let's say for me, sorry,

pipeline. And let's say for me, sorry, for me here,

for me here, if I were to run like a pipeline,

if I were to run like a pipeline, let's say next row run.

let's say next row run. >> I think you've forgotten a

>> next. Oh yes, right. next and of course I will pin a version

next and of course I will pin a version using minus R wait I don't remember

using minus R wait I don't remember which is the latest version of

And by using like this minus r command, I'm really pinning uh this pipeline to

I'm really pinning uh this pipeline to be used like to use a specific version

be used like to use a specific version of the pipeline that will correspond to

of the pipeline that will correspond to the tag and that way it works. Next flow

the tag and that way it works. Next flow automatically pull the pipeline. I have

automatically pull the pipeline. I have never run this pipeline on this computer

never run this pipeline on this computer yet because I just reinstalled my

yet because I just reinstalled my computer recently. So everything just

computer recently. So everything just happened. Uh next row he will be using

happened. Uh next row he will be using like in that particular case docker to

like in that particular case docker to pull in all of the images that we need

pull in all of the images that we need to run this pipeline and then the wall

to run this pipeline and then the wall pipeline run and like everything is

pipeline run and like everything is happening and my own analysis is done.

happening and my own analysis is done. This is just running a test profile. It

This is just running a test profile. It has like small data so that it should

has like small data so that it should run like more or less fast.

run like more or less fast. >> Amazing.

>> Amazing. >> This is what we use for RCI.

>> This is what we use for RCI. >> One last question and then I will really

>> One last question and then I will really end this even though I don't want to

end this even though I don't want to because we did say 90 minutes. um

because we did say 90 minutes. um uh you have like a fairly developed

uh you have like a fairly developed governance structure and I'm curious

governance structure and I'm curious about um when that developed and if it

about um when that developed and if it developed in response to any sort of

developed in response to any sort of impetus in particular

impetus in particular >> uh I think we so basically um Phil u has

>> uh I think we so basically um Phil u has created like multiqc which is a python

created like multiqc which is a python package that uh python tool Yes, I'm bad

package that uh python tool Yes, I'm bad with Python, so excuse me if I'm like

with Python, so excuse me if I'm like not speaking the right term there. Uh

not speaking the right term there. Uh that basically

that basically collect all of the reports, all of the

collect all of the reports, all of the metrics that you could get outside of a

metrics that you could get outside of a tool and generize for you uh a final

tool and generize for you uh a final report that we will say a multi report.

report that we will say a multi report. It's one of the most common tool in

It's one of the most common tool in bioinformatics because most of the tool

bioinformatics because most of the tool produce metrics and everyone is

produce metrics and everyone is interesting by seeing metrics to see how

interesting by seeing metrics to see how your experiment has gone. So the idea is

your experiment has gone. So the idea is fairly uh simple fairly stupid but super

fairly uh simple fairly stupid but super efficient because it works well. It's

efficient because it works well. It's what everyone wants. So this tool has

what everyone wants. So this tool has been one of the main mainly used tool

been one of the main mainly used tool everywhere and I think Phil has seen

everywhere and I think Phil has seen what happened with this particular

what happened with this particular project.

project. and he decided that he didn't want to be

and he decided that he didn't want to be the main responsible for that and at the

the main responsible for that and at the very beginning NF core was created with

very beginning NF core was created with cy administrator I think the other team

cy administrator I think the other team came in like a tiny bit by tiny bit uh

came in like a tiny bit by tiny bit uh when they were needed and um

when they were needed and um and yes and we still we are still

and yes and we still we are still creating new team because we just

creating new team because we just created the doc the team dedicated to

created the doc the team dedicated to documentation

documentation >> um when would you When would you say

>> um when would you When would you say that change happened? Like what year

>> uh about what the teams? >> Yeah, I guess when you were talking

>> Yeah, I guess when you were talking about like uh when did the report start

about like uh when did the report start getting created and when did

getting created and when did >> Oh. Oh no. So like you see that was a

>> Oh. Oh no. So like you see that was a previous project from field. So I think

previous project from field. So I think that started like many years ago. It's

that started like many years ago. It's completely disconnected from NFC like

completely disconnected from NFC like it's not completely disconnected. It's

it's not completely disconnected. It's like we use it as as the final tool for

like we use it as as the final tool for most of our like for most of our

most of our like for most of our pipeline because most of the pipeline

pipeline because most of the pipeline are using the tools that can be used to

are using the tools that can be used to feed into mult. So for us we just use

feed into mult. So for us we just use that as a final report that you will

that as a final report that you will give to your user or you will look

give to your user or you will look yourself and you see okay this

yourself and you see okay this sequencing quality is good or so on.

sequencing quality is good or so on. >> Gotcha. Okay. Sorry, I think I

>> Gotcha. Okay. Sorry, I think I misunderstood

misunderstood the run.

the run. >> Yeah, what I was trying to say is that

>> Yeah, what I was trying to say is that is that Phil had experiment with a

is that Phil had experiment with a previous project that he knew how hard

previous project that he knew how hard it was to handle such a big project by

it was to handle such a big project by himself. I think he knew that this

himself. I think he knew that this project was going to be huge and decided

project was going to be huge and decided right away at the beginning, okay, we

right away at the beginning, okay, we are going to be like a team of people

are going to be like a team of people that u work together and we can do stuff

that u work together and we can do stuff together and we collaborate and we all

together and we collaborate and we all take turn into doing things together.

take turn into doing things together. >> I think it'd be interesting. I don't

>> I think it'd be interesting. I don't know if you have this like some sort of

know if you have this like some sort of like graph or map of like the number of

like graph or map of like the number of participants like over time. Um because

participants like over time. Um because I think relatively

I think relatively >> relatively rare for project founders to

>> relatively rare for project founders to have that level of insight around

have that level of insight around governance. I feel like the com far more

governance. I feel like the com far more common model is someone I guess like

common model is someone I guess like Phil experienced with that first project

Phil experienced with that first project of like taking on something by yourself

of like taking on something by yourself and then being like oh no I'm burning

and then being like oh no I'm burning out or this is too much for me and only

out or this is too much for me and only then sort of trying to figure out a a

then sort of trying to figure out a a different governance structure.

different governance structure. >> So let me check. I know that a colleague

>> So let me check. I know that a colleague of mine we we've been like really

of mine we we've been like really talking about that like at the last ret

talking about that like at the last ret and we've been working on some um

>> I feel like you guys really visualization so

visualization so >> yeah we've been working on some

>> yeah we've been working on some visualization uh so the vation is still

visualization uh so the vation is still like we we need to work more on the

like we we need to work more on the visation so let me try to share my

visation so let me try to share my screen one more time screen share

Okay. Yes. Oh, that's the one. Okay. So, basically we can see that people

So, basically we can see that people were like we there was a couple of us at

were like we there was a couple of us at the beginning and then uh

the beginning and then uh this is how many we are at the moment

this is how many we are at the moment and some people join like for for

and some people join like for for smaller term. And yes, basically this is

smaller term. And yes, basically this is more or less what we what we know about

more or less what we what we know about the people that were part of the of the

the people that were part of the of the core team that administrate Fore.

core team that administrate Fore. >> Cool. It sounds it seems like you had

>> Cool. It sounds it seems like you had like a a a whole group of people to

like a a a whole group of people to start with and then a few successive

start with and then a few successive waves of people coming in. Um and then

waves of people coming in. Um and then overall overall you've grown even though

overall overall you've grown even though individual people have moved on.

>> Yeah. Some people moved on because they either like uh join some other company,

either like uh join some other company, they switch job,

they switch job, they

they change completely like uh Kevin for

change completely like uh Kevin for example like switch fields completely.

example like switch fields completely. So he's no longer like working in

So he's no longer like working in binformatics. So that's why he's he's

binformatics. So that's why he's he's now an alumni. Uh

now an alumni. Uh UNS felt like this project was too big

UNS felt like this project was too big for him so he didn't join. Andreas was

for him so he didn't join. Andreas was one of the founder with Phil and Alex,

one of the founder with Phil and Alex, but unfortunately he wasn't able like to

but unfortunately he wasn't able like to to continue forward with us. So

to continue forward with us. So he he dropped pretty early.

he he dropped pretty early. Yes, Alex left because it was too

Yes, Alex left because it was too difficult for him with his new position.

difficult for him with his new position. Yeah. And I mean all of those are pretty

Yeah. And I mean all of those are pretty normal reasons. And I think it's really

normal reasons. And I think it's really great that you've set up a system and a

great that you've set up a system and a community where any one person can take

community where any one person can take a step back if various life or career

a step back if various life or career related reasons make it not a fit for

related reasons make it not a fit for them.

them. >> As opposed to there's so many projects

>> As opposed to there's so many projects that are so reliant on one person that

that are so reliant on one person that that person feels that they need to

that person feels that they need to continue even if they personally their

continue even if they personally their needs and their desires no longer match

needs and their desires no longer match up with the project. So I really

up with the project. So I really appreciate your structure.

appreciate your structure. >> Usually what we try to do, we try to

>> Usually what we try to do, we try to list like for each particular team what

list like for each particular team what are the leads, what are the member and

are the leads, what are the member and people that used to be like uh involved

people that used to be like uh involved in the team and yes like the maintenance

in the team and yes like the maintenance I think is one of the biggest team but I

I think is one of the biggest team but I think for us what is yes this is all

think for us what is yes this is all open source so no one is u

open source so no one is u like we don't own any money ourself.

like we don't own any money ourself. So I think it's quite important and uh

So I think it's quite important and uh yes uh we we did manage to get some uh

yes uh we we did manage to get some uh funding from the trans carbag initiative

funding from the trans carbag initiative years ago but yes that's no longer

years ago but yes that's no longer possible

possible unfortunately

unfortunately >> most of the people are most of the

>> most of the people are most of the people who contribute they're getting

people who contribute they're getting funding through their jobs or their

funding through their jobs or their graduate students or uh professors.

graduate students or uh professors. >> Yeah. Yeah. Basically uh let's say like

>> Yeah. Yeah. Basically uh let's say like currently um I have contributors that

currently um I have contributors that are working with me on some pipeline. Uh

are working with me on some pipeline. Uh basically it will be it will be there

basically it will be it will be there doing that because uh they figured it

doing that because uh they figured it was easier for them to collaborate with

was easier for them to collaborate with us on uh tiny part on that project than

us on uh tiny part on that project than to develop the world stuff on their

to develop the world stuff on their side. So it's more okay let's contribute

side. So it's more okay let's contribute to to an open source stuff other than

to to an open source stuff other than let's just do stuff like internally.

let's just do stuff like internally. >> Yeah. So in some sense most of the

>> Yeah. So in some sense most of the people who are contributing are paid

people who are contributing are paid contributors in the sense that it's part

contributors in the sense that it's part of a job that financially supports them

of a job that financially supports them and it's not like completely unrelated

and it's not like completely unrelated to their financial support volunteer

to their financial support volunteer work. Um but they're not getting paid by

work. Um but they're not getting paid by you directly. It's a collaborative

you directly. It's a collaborative >> it's a collaborative process. Yeah.

>> it's a collaborative process. Yeah. because we like we get some funding from

because we like we get some funding from uh some private company to help us um

uh some private company to help us um especially with our infrastructure. Um I

especially with our infrastructure. Um I think um I don't remember who is paying

think um I don't remember who is paying for Slack but someone is paying for our

for Slack but someone is paying for our Slack. AWS is helping us a lot with our

Slack. AWS is helping us a lot with our infrastructure. We have like our runners

infrastructure. We have like our runners for the CI are being paid by AWS and

for the CI are being paid by AWS and also we do some bigger testing and

also we do some bigger testing and they're helping us with that. Sec

they're helping us with that. Sec contributing quite heavily as well in

contributing quite heavily as well in term of people that work from time to

term of people that work from time to time on that and then it really depends

time on that and then it really depends on like what people are doing or not. I

on like what people are doing or not. I think we are starting to see maybe a bit

think we are starting to see maybe a bit more of like uh freelancers or um

more of like uh freelancers or um or just like paid contributors that are

or just like paid contributors that are paid by a company to work on specific

paid by a company to work on specific project and I think the field might in

project and I think the field might in evolve evolve into that that direction.

evolve evolve into that that direction. Do you have any issues with um I've

Do you have any issues with um I've sometimes seen this in projects I've

sometimes seen this in projects I've participated in or just advised on or

participated in or just advised on or just viewed from afar where

just viewed from afar where when you have a model where people are

when you have a model where people are basically being paid by their work to

basically being paid by their work to contribute

contribute um sometimes that can lead to issues

um sometimes that can lead to issues where the things that their company is

where the things that their company is sort of pushing them to work on are

sort of pushing them to work on are things that are directly related to that

things that are directly related to that company's success. And so things like

company's success. And so things like for instance community management or

for instance community management or governance or um

governance or um like some stuff can get overlooked

like some stuff can get overlooked because it's either more infrastructural

because it's either more infrastructural or like quote unquote like soft uh stuff

or like quote unquote like soft uh stuff that's actually really important to the

that's actually really important to the community but not it's harder to make a

community but not it's harder to make a case to your manager or whatever that

case to your manager or whatever that this is an effective use of your

this is an effective use of your >> Yeah. No, no, definitely definitely like

>> Yeah. No, no, definitely definitely like I think we are lacking people like on in

I think we are lacking people like on in outreach and

outreach and documentation

documentation because yes it's hard to u to it's hard

because yes it's hard to u to it's hard to explain to anyone that is paying you

to explain to anyone that is paying you how important that is and explaining to

how important that is and explaining to anyone that okay my contribution to this

anyone that okay my contribution to this open source process will be useful for

open source process will be useful for like tons of people

like tons of people is not really interesting like uh the

is not really interesting like uh the people that are paying me or not like

people that are paying me or not like like it's yeah I might be working a lot

like it's yeah I might be working a lot for for like this organization that is

for for like this organization that is completely like not profit but yes the

completely like not profit but yes the people I work for like should profit

people I work for like should profit from stuff so yes it's bit hard but yes

from stuff so yes it's bit hard but yes I think the people that are working at

I think the people that are working at my level are passionate people and yes

my level are passionate people and yes we're having this uh talk it's 8:30

we're having this uh talk it's 8:30 for me so

for me so >> yeah

>> yeah you can see like yes if I do this kind

you can see like yes if I do this kind of stuff it's

of stuff it's >> a bit outside of my work hour and I'm

>> a bit outside of my work hour and I'm used to that because we collaborate with

used to that because we collaborate with people all around the globe so I have

people all around the globe so I have colleagues in Canada in New Zealand so

colleagues in Canada in New Zealand so it's hard to find time to work together

it's hard to find time to work together >> yeah well I definitely appreciate take

>> yeah well I definitely appreciate take you taking time to talk with me for over

you taking time to talk with me for over an hour and a half about your project um

an hour and a half about your project um and I can see from

and I can see from um the documentation the the structure

um the documentation the the structure of the bot, like all of these like

of the bot, like all of these like thoughtful ways that you've um

thoughtful ways that you've um like really been taking a sort of global

like really been taking a sort of global collaborative perspective. Um and it

collaborative perspective. Um and it sounds like that was something that was

sounds like that was something that was part of the culture from the beginning

part of the culture from the beginning with the founder sort of being like um

with the founder sort of being like um let's not make this all about one

let's not make this all about one person's needs or one person's putting

person's needs or one person's putting it all on one person's shoulders, but

it all on one person's shoulders, but instead making it a a group effort. Um

instead making it a a group effort. Um yeah, uh this was really interesting.

yeah, uh this was really interesting. Um, I've never done one of these before,

Um, I've never done one of these before, so I don't entirely know how to wrap

so I don't entirely know how to wrap this up. Um, is there anything else you

this up. Um, is there anything else you wanted to add or say? Um,

wanted to add or say? Um, >> um,

no, I think it Yes, I Yes. No, I should have like presented like more stuff that

have like presented like more stuff that we are doing, but yes, we don't have

we are doing, but yes, we don't have enough time. But yes, definitely what

enough time. But yes, definitely what what is fun as well is part of the stuff

what is fun as well is part of the stuff that we're doing on the outreach side

that we're doing on the outreach side will be like the byite size talks that

will be like the byite size talks that we are doing on the next floor side. I

we are doing on the next floor side. I know that the SEA community team is

know that the SEA community team is doing some podcast which are like quite

doing some podcast which are like quite interesting. So yes, we are

interesting. So yes, we are collaborating a lot with each other and

collaborating a lot with each other and we are collaborating also with other

we are collaborating also with other community. Uh so yes no for me I think

community. Uh so yes no for me I think like what is key for me into all of that

like what is key for me into all of that is trying to be as open as possible as

is trying to be as open as possible as transparent as possible try to

transparent as possible try to contribute all together. Um and yes

contribute all together. Um and yes because all of that goes toward like the

because all of that goes toward like the goal of being reproducible. So yes for

goal of being reproducible. So yes for me these three concept are what drives

me these three concept are what drives NFCO and what drives my my work every

NFCO and what drives my my work every day.

day. >> Excellent. Well, if you have anything

>> Excellent. Well, if you have anything else you want to add or any links or

else you want to add or any links or whether it's like your podcast stuff,

whether it's like your podcast stuff, etc., at some point, hopefully soon, I

etc., at some point, hopefully soon, I will um I got to make a YouTube and um

will um I got to make a YouTube and um probably like a Fedverse Tube channel.

probably like a Fedverse Tube channel. Is it Pure Tube? I think it's Pure Tube.

Is it Pure Tube? I think it's Pure Tube. Um

Um >> Pier.

>> Pier. >> Uh so, I'll make a channel and I will

>> Uh so, I'll make a channel and I will post this video at some point. Um but I

post this video at some point. Um but I haven't actually made the channel so I

haven't actually made the channel so I can't like post it immediately. But in

can't like post it immediately. But in any case, um if you have any links um

any case, um if you have any links um beyond like I can definitely get the

beyond like I can definitely get the link to your website, but if you have

link to your website, but if you have more specific things like a particular

more specific things like a particular like like new contributor area of the

like like new contributor area of the docs or like a ways for people to learn

docs or like a ways for people to learn more, just things you want to highlight,

more, just things you want to highlight, I'm happy to like make the description a

I'm happy to like make the description a bunch of links. Uh

bunch of links. Uh >> yeah, no, no, definitely. I will anyway

>> yeah, no, no, definitely. I will anyway I think I I will um

I think I I will um go back with my like um contributors

go back with my like um contributors like within the core team. Uh let them

like within the core team. Uh let them know what we did and um how we do stuff

know what we did and um how we do stuff and see if they want like to add

and see if they want like to add anything because yes, I probably like

anything because yes, I probably like forgot tons of stuff.

forgot tons of stuff. >> I also gave you absolutely no structure

>> I also gave you absolutely no structure to this. Hopefully once I do a couple of

to this. Hopefully once I do a couple of these I'll get a sense for the kinds of

these I'll get a sense for the kinds of things that is helpful to talk about and

things that is helpful to talk about and then I can be like hey prepare to talk

then I can be like hey prepare to talk about these things. Uh but I for anyone

about these things. Uh but I for anyone viewing this in the audience I gave you

viewing this in the audience I gave you absolutely like no guidance because I

absolutely like no guidance because I didn't know what we were going to talk

didn't know what we were going to talk about because it's the first one. So you

about because it's the first one. So you know

know >> no but that it was super fun and I think

>> no but that it was super fun and I think it was good to have like such an

it was good to have like such an unguided like structure and yes

unguided like structure and yes >> I had no slide prepare. I just used like

>> I had no slide prepare. I just used like the the website. Oh, no. One last stuff

the the website. Oh, no. One last stuff that I really want to to show. Uh, let

that I really want to to show. Uh, let me just see if I have that somewhere.

Uh, it's sad that you cannot see my computer at the moment. Uh,

because the stuff I want to show is >> You can share your screen again if you

>> You can share your screen again if you want. No, no, but like it's not my Yeah,

want. No, no, but like it's not my Yeah, I will share my screen, but that's not

I will share my screen, but that's not my screen that I wanted to share.

my screen that I wanted to share. >> Sorry. Sorry. Right.

>> Okay, this is my computer. >> Oh, it's not showing it. Oh,

>> Oh, it's not showing it. Oh, >> okay. Sorry.

>> okay. Sorry. >> Oh, there it is. Sorry. I'm I'm just too

>> Oh, there it is. Sorry. I'm I'm just too impatient. Oh. Oh, I love that.

impatient. Oh. Oh, I love that. >> This is my computer. We make stickers

>> This is my computer. We make stickers quite a lot.

quite a lot. >> Ask about that because you have a drop

>> Ask about that because you have a drop down that's like it's like maybe it's

down that's like it's like maybe it's the about section. It's like governance

the about section. It's like governance and then the next one is stickers. And I

and then the next one is stickers. And I was like stickers. Why are stickers at

was like stickers. Why are stickers at the same level as governance? Yes,

the same level as governance? Yes, stickers because we are we are like

stickers because we are we are like highly into stickers. So this is S. This

highly into stickers. So this is S. This is like my pipeline. So I designed the

is like my pipeline. So I designed the sticker. I printed it myself. These are

sticker. I printed it myself. These are like the golden sticker are the ones

like the golden sticker are the ones that we make for specific event. So we

that we make for specific event. So we had one for some quiz. when we meet each

had one for some quiz. when we meet each other uh we uh yeah when we meet each

other uh we uh yeah when we meet each other in Akaton we add like stickers of

other in Akaton we add like stickers of socks around and the people that find

socks around and the people that find the most so like get sticker for that we

the most so like get sticker for that we made a tiny video game these are part of

made a tiny video game these are part of the team that I'm part of so outreach

the team that I'm part of so outreach mainteners infrastructure and these are

mainteners infrastructure and these are like all of the events that have been

like all of the events that have been happening in the community I've been a

happening in the community I've been a mentor I'm a network ambassador as well

mentor I'm a network ambassador as well and yes so we've been trying to to do

and yes so we've been trying to to do stuff. And we do print other like

stuff. And we do print other like specific stickers uh like merge co we

specific stickers uh like merge co we have like documentation

robot or like so yeah we try to be like fun and like have this kind of stuff and

fun and like have this kind of stuff and at every event that you go you will have

at every event that you go you will have such stickers.

such stickers. >> I love this so much. That's so cute. Um,

>> I love this so much. That's so cute. Um, >> and I think I think my colleague will be

>> and I think I think my colleague will be proud that I did not forget about the

proud that I did not forget about the stickers.

stickers. >> Excellent. All right, I'm going to go

>> Excellent. All right, I'm going to go ahead and stop the recording. So,

ahead and stop the recording. So, goodbye to anyone who's watching this.

goodbye to anyone who's watching this. See you next time.

Click on any text or timestamp to jump to that moment in the video

Most transcripts ready in under 5 seconds

One-Click Copy125+ LanguagesSearch ContentJump to Timestamps

Paste YouTube URL

Enter any YouTube video link to get the full transcript

Most transcripts ready in under 5 seconds

Get Our Chrome Extension

Get transcripts instantly without leaving YouTube. Install our Chrome extension for one-click access to any video's transcript directly on the watch page.

Add to Chrome — Free

Works with YouTube, Coursera, Udemy and more educational platforms

Get Instant Transcripts: Just Edit the Domain in Your Address Bar!

YouTube

←

→

↻

https://www.youtube.com/watch?v=UF8uR6Z6KLc

YoutubeToText

←

→

↻

https://youtubetotext.net/watch?v=UF8uR6Z6KLc

YouTube TranscriptPreparing your results…

YouTube Transcript:Getting to Know NF-Core (Open Source Project Tours)