2024 Collaboration Cafe Notes Archive#
2024-11-19#
Check-in#
Min / @minrk / Simula
Samuel / @sgaist / Idiap Research Institute (might be a couple minutes late)
Simon / @manics / University of Dundee
Jonah / @jduckles / Organizational Mycology
Dan / @dsholler / Organizational Mycology
Erik / @consideRatio / 2i2c
Sarah / @sgibson91 / 2i2c
Yuvi / @yuvipanda / 2i2c
Beth / @bduckles / Organizational Mycology
Celebrations and Shout-Outs :tada:#
Z2JH 4.0.0 is out!
TLJH 2.0.0 is out!
Agenda#
Min (10m): Adopting JupyterHealth into JupyterHub
Proposal to formally bring this project under Jupyter governance, hopefully submitted in a couple of weeks
Related to a TRE-integrated JH? Related in principle but perhaps not exactly the same
Yuvi (5min): 2i2c & Structured Open Source Policy for getting more people involved in upstream maintenance in JupyterHub
https://hackmd.io/rteifFE0SV6Hk20KRfMR0w and https://hackmd.io/k_gIv0_uRa6yRXsNDnFLfQ?view and https://hackmd.io/-3p4o-SITM668FIGKn-2SQ?view
Please provide feedback on what you think! (as comments in the doc or email to me at yuvipanda@2i2c.org)
OrgMycology and Turing Way (10 min): Kicking off the “Voices of JupyterHub” project interviews the-turing-way/jupyterhub-orgmyc-EOSS Link to express interest in being interviewed: Click Here
Advertise this on Discourse for greater reach? https://discourse.jupyter.org/c/jupyterhub/10
Min (10 min): JupyterHub images jupyterhub/jupyterhub#4954
Simon (10 min): Integrating jhsingle-native-proxy
Simon (10 min): JupyterHub TREs
Partly related to lsc-sde/lsc-sde#55
2024-10-15#
No meeting due to lack of attendees.
2024-09-17#
Check-in#
Min / @minrk / Simula
Samuel Gaist / @sgaist / Idiap Research Institute (might be 5 min late)
Sarah Gibson / @sgibson91 / 2i2c
Erik Sundell / @consideratio / 2i2c
Arnim Bleier /@arnim / GESIS
Dan Sholler / @dsholler / Organizational Mycology
shane knapp / @shaneknapp / UC Berkeley
Kirstie Whitaker / @KirstieJane / Alan Turing Institute
Raniere Silva / @rgaiacs / GESIS
Celebrations and Shout-Outs :tada:#
Erik (@consideRatio) doing lots of maintenance and update on ldapauthenticator and others. Thank you!
notebook 7 now on mybinder.org
Agenda :clock2:#
Min (10m): spreading meeting facilitation responsibilities: jupyterhub/team-compass#737
Maybe bring it up on the forum
wider advertisement of meetings, e.g. forum. Most things are only on Issues right now
relying on volunteers leads to bias in who participates
Sarah offers mentorship for facilitators
Broader question: Standing agenda item on community engagement/activity, who is coming or not, etc.
Nominate a person to make a list of where to share things (write here):
Jupyter forum
Twitter? / Mastodon? Do we have those accounts anymore?
Gitter?
Dan Sholler (5 minutes): Voices of Jupyter Research Project with The Turing Way
Scope of Work here: https://docs.google.com/document/d/1M2kfwvT7KHfgWd2Qph3BIM7seyrb8Lt23cVE_1HzvMw/edit#heading=h.x5kwcnqrf9m2
Track progress and contribute here: the-turing-way/jupyterhub-orgmyc-EOSS
Past report as a sample: https://astropy-dei.orgmycology.com/
Arnim c/o Min (10m): discuss GESIS in mybinder.org
nebulous “issues” launching repos that don’t have local images
local image cache limited to 2TB, if it needs to be pulled from docker hub lots of traffic and sometimes very slow
If pull fails due to timeout, this triggers a new build
short-term: Lower GESIS role in federation to avoid timeouts
“passthrough” registry docker configuration to reduce push/pull roundtrip all the way to docker hub
Shane (no idea how much time): Configurable HTTP proxy running out of ephemeral ports
happens when ~300+ users are logged and working simultaneously
massive user impact
-
The default ephemeral port range contains more than 28,000 ports (60999+1-32768=28232). Does that mean we can have at most 28,000 outgoing connections? That’s the core question of this blog post!
do we limit number of users per hub (will be ‘difficult’ for some course-specific hubs with 1200 users)?
somehow load-balance the chp (a bit meta lol)
Proposal to try switching to traefik again: jupyterhub/zero-to-jupyterhub-k8s#3497
2024-08-20#
Check-in#
Sarah Gibson / / 2i2c
Min / @minrk / Simula
Erik / @consideratio / 2i2c
Celebrations and Shout-Outs#
Min did amazing work in a KubeSpawner PR!
Simon plans on releasing Traefik Proxy this week
Agenda#
Erik (10m): KubeSpawner project complications in not-officially maintained KubeIngressProxy?
What’s the state of this meeting?
Sarah hasn’t been in the Community Strategic Lead role for a while now, so faciliatating this meeting should not by default fall to her
Spreading the load: roster or volunteers, another mechanism?
Sarah would appreciate someone else to begin facilitating the US TZ meeting as that can run quite late into the evening for UK
Does the format of this meeting still work? Reduce back to 1hr? Maybe more frequent and shorter meetings?
Maybe more specific meetings, e.g., maintainers office hours, governance board strategy meeting, as well as free(er)-form community meetings
Sarah would also like some help revisiting some work done by an external governance consultant: https://docs.google.com/document/d/1WJMRR7ggIivmccrAjIgFWwLt9NYpzrn87ku-4GAFbnw/edit?usp=sharing Please reach out to her if you would like to pair on a specific proposal
Erik has volunteered to pair on Proposal A: jupyterhub-contrib org
2024-07-17#
Check-in#
Sarah Gibson / sgibson91 / 2i2c
Diogo Castro / diocas/ CERN
Simon Li / manics / University of Dundee
Samuel Gaist / @sgaist / Idiap Research Institute
Min RK / minrk / Simula
Agenda#
Samuel (10m): Kubespawner -> operator discussion
SwissDataScienceCenter/amalthea is becoming Jupyter agnostic
Samuel (10m): Multispawner
-
-
Should no (or same) prefix be allowed for multiple authenticators? Definitely not by default.
Possibly through configuration.
-
Maybe prefix should never modified by normalisation?
-
PyPI: https://pypi.org/project/jupyterhub-multiauthenticator/
Not the same as https://pypi.org/project/multiauthenticator/
-
CodeSpell PRs:
A few PRs have been opened, should we roll it out across JupyterHub?
2i2c uses it, pre-commit only (no GitHub workflow needed)
2024-06#
Meeting cancelled due to folk on vacation.
2024-05-22#
Check-in#
Sarah / @sgibson91 / 2i2c
Min / @minrk / Simula
Erik / @consideratio / 2i2c
Simon / @manics / University of Dundee
Blessing / @bl-aire
Samuel / @sgaist / Idiap Research Institute
Shaun / @qcaas-nhs-sjt / NHS Lancashire and South Cumbria ICS
Celebrations and Shout-Outs#
Thanks to Michał Krassowski for landing authenticator-managed Roles: jupyterhub/jupyterhub#4748
Agenda#
Min (5min): JupyterHub 5 release ready
Min (10min): repo2docker updating to JupyterLab 4, adding Python 3.12
lab 4 doesn’t support 3.7 jupyterhub/repo2docker#1340
No objections to freezing 3.7 environment
Add Python 3.12 jupyterhub/repo2docker#1347
Can we decide up front what to do if a frozen environment breaks so we don’t waste time investigating?
Difficult, since if it breaks we still need to investigate whther it’s a true breakage, or a temporary failure
Erik (5min): Releases of dependencies for z2jh and tljh to go with JupyterHub 5 bump.
Help needed with reviewing potential PRs (or making any other breaking changes):
Simon/Shaun (5min): Secure remote desktop access with JupyterHub (Trusted Research Environments)
Shaun (5min): Unifying Kubespawner with the Kubernetes Controller Model
Min: Might be better to start from scratch, and copy any useful code from KubeSpawner
Other related work from Swiss Data Science Center:
Ryan (5 min): Hub with named servers. Students and TAs (teaching assistents).
TAs have access to student servers and home directories(?)
Some courses reverse roles though (TA in one course is student in another and vice-versa)
Is there a way to grant role permissions based on server name (e.g. course prefix)?
Simon (5 mins): Latest release of
requestsbreaks docker-py, which breaks lots of our CI and any apps using docker-py e.g. repo2docker, chartpress
2024-04#
Meeting cancelled due to low attendance.
2024-03-19#
Check-in#
Min / @minrk / Simula
Sarah Gibson / @sgibson91 / 2i2c
Mike Krassowski / @krassowski / Quansight
Erik / @consideratio / 2i2c
Celebrations and Shout-Outs#
Thanks to Erik, assisted by Yuvi, for working on jupyter-server-proxy security patch
Agenda#
Min (10 min): Preparing JupyterHub security release jupyterhub/jupyterhub
Erik misc releases + security advisories
[x] batchspawner, jupyterhub/batchspawner#302
[ ] oauthenticator
The ask: help review advisory and patch PR, and provide agreement/disagreement on going for a release ~today
Changelog PR: jupyterhub/oauthenticator#732
security advisory: jupyterhub/oauthenticator
security advisory PR: jupyterhub/oauthenticator-ghsa-55m3-44xf-hg4h#2
[ ] jupyter-server-proxy
the ask: OK to publish advisory today?
releases are already made, but not with changelog yet
security advisory to be published - today!?: jupyterhub/jupyter-server-proxy
Mike JupyterHub release timelines (ballpark),
manage_rolesPR (jupyterhub/jupyterhub#4050)Erik Backport release in conda-forge? Anyone with experience?
Agenda item goal: understand if we go for that
2024-02-20#
Check-in :raising_hand:#
Min / @minrk / Simula
Sarah / @sgibson91 / 2i2c
Samuel / @sgaist / Idiap Research Institute
Kirstie / @KirstieJane / The Alan Turing Institute
Simon / @manics / University of Dundee
Steve / @stevejpurves / Curvenote | MystMarkdown
Introduce yourself! :wave:#
If you are new to the meeting, add your name below and you can introduce yourself at the start of the meeting.
Kirstie Whitaker - Programme Director for Tools, Practices and Systems at The Alan Turing Institute. Founder of The Turing Way. Very interested in open source governance (EDI, accountability, transparency, project management etc)
Agenda :clock2:#
Kirstie (15mins): CZI Essential Open Source Software for Science (EOSS) Diversity, Equity and Inclusion (DEI) funding revamp proposal
https://docs.google.com/document/d/1pSU9kG_XDOSTF4JcbIt18hep1rCHIjuUYuYi5mg0q-0/edit?usp=sharing
Sarah found Outreachy is not well suited to the capacity of JupyterHub’s community
There isn’t a dedicated leadership meeting for the council
Proposal:
develop community engagement strategy and identify contributor blockers by speaking with the leadership council
not building a specific community development guide and create clearer x-refs between Jhub team compass and The Turing Way building on Jhub community’s needs
building trainers for community members to mentor new contributors
embedding inclusive practices into Jhub community
Questions?:
How to address the issue of feeling empowered to do stuff (Kirstie specifically?) Where to do the work (project is spread out), lack of clarity and explicitness makes things difficult, make sure we’re raising issues on what is unclear, raising things and not getting a response, escalation process to ensure Kirstie has support
KW will come along to collab cafe regularly to report back and escalate
It’s a feature to identify the barriers
KW wants to know how accurate is the membership of the jhub council? Can anyone identify any risks of talking to the council about what it means to lead the project? E.g. “Don’t do X”
Variety of interaction in the council, and there isn’t a clear set of expectations in what the council is doing
different types of leadership, technical decisions, community management, organisational leadership, paid development time
Steve (15mins): Contributing a new content/repo provider (MECA) to BinderHub/mybinder.org - how to?
Notebooks Now! project run by AGU, funded project to pilot using Notebooks to build interactive, computational article, and formally publish them
How to integrate with XML format (specifically JATS used by journals, also comes with a zip file MECA) so that Notebook-based publications can be accepted
MECA is an existing NISO standard (https://www.niso.org/standards-committees/meca) , modify JATS to also understand Notebooks, and bundle MECA with the binder requirements for launching. Completely disassociate from a GitHub repo.
https://agu.curve.space - demo website showing what the journal might look like
New repo-provider “MECA for binder”, how to get code into repo2docker? how to deploy it onto mybinder.org?
Kirstie (15mins): “I think this is the first barrier for you Kirstie 😉 A lot of folk turn up to this meeting with a “I have a PR, how do I push it forward?” style question”
We don’t have much of a review process at the moment and it is hard to allocate time and expertise to the review.
Do we make the decision in the meeting? How many people need to say “yes” for it to be a real “yes”!
Process isn’t written down.
The project is very technical and there are a lot of information siloes.
Some folks don’t feel confident enough to review (and that might be truely that they don’t have the expertise! Even if they’re leaders in the community.)
Identifying reviewers in advance can be hard.
Some other projects allocate a reviewer (often by a bot) and then they know that it is their responsibility to take the review forwards.
They become the point person so even if they don’t know the right details they can go and get the person who has the expertise.
Additional challenge around repo2docker as a specification - might have additional maintenance burden if edits are incorporated there.
Bias towards the more recent tasks and the easier or noiser tasks for reviewers.
Potentially needs a specific meeting (or async equivalent) to set the priorities
Hard to figure out how to reach consensus
“Yuvi and Sarah spoke to a governance expert on some of these things, I think section C here is particularly relevant”: https://docs.google.com/document/d/1WJMRR7ggIivmccrAjIgFWwLt9NYpzrn87ku-4GAFbnw/edit?usp=sharing
No actions taken yet.
Could try a bug triage activity - maybe part of the collaboration cafe on a regular basis.
Tried in the original vision but didn’t get momentum yet: https://blog.jupyter.org/online-collaboration-café-launch-jupyterhub-team-meetings-to-become-more-collaborative-spaces-b713edadf15
Maybe not enough people yet in the meetings
Monthly meetings mean that there are big chunky topics to discuss so again triaging drops down the priority list.
Maybe more often? Maybe not trying to do everything in the same meeting (collaboration cafe)??
Hard with different time zones! (A wicked problem, no easy answer!)
2024-01-16#
Check-in#
Min / @minrk / Simula
Samuel / @sgaist / Idiap Research Institute
Simon / @manics / University of Dundee
Raniere / @rgaiacs / GESIS
Erik / @consideRatio / 2i2c
Agenda#
Min (15m): user-initiated sharing PR is complete and ready for review
key points for review:
architecture overview
permissions
defaults
pydantic v2 dependency
questions:
samuel: is a share-code single use or usable multiple times to grant users permissions?
min: its re-usable, and state is kept on how many times its used.
samuel: it would be a relevant feature to let a share code be usable once and only once
min: current inclination is to not provide once-use codes yet, but its not that hard to support
erik: when this is released, what a implication?
admins: enabling sharing
users: tricky but possible to generate share links
min: reference + tutorial + example is provided
erik: review scrutiny
review in scopes.py file is relevant to ensure stability, there is some refactoring related to “has scopes” which is used in “am i allowed” handlers.
there is now public api where it was previously private
min: besides this, there is review of relevance about granularity of scope and missing features
Min (15m): JupyterHub 5 planning
PRs awaiting review:
(the prs can be reviewed separately, where /hub/info should be at least reviewed by Yuvi who requested the feature)
Features implemented, awaiting release:
Min (10m): Security discussion
Samuel (10m): repo2podman ignore file management
PRs awaiting reviews
Points that needed some clarifications:
The ignore files are to be used at the Docker level and not repo2docker itself
The goal is to make repo2docker work in a similar fashion to podman and docker where files listed there will not be sent to the context and thus not part of the image
Possible issues:
May current repository break if these files are now being actively used ? It’s likely going to be a rare case.
The documentation should be improved to make clear that the ignore file do not apply to repo2docker itself.
Simon (3m) BinderHub versioning
CalVer is hard to get out of: https://jacobtomlinson.dev/posts/2023/sometimes-i-regret-using-calver/
Erik (3m) Z2JH breaking change merged requiring k8s 1.24+, and in end of january also k8s 1.25+ - I’m thinking we wait for jupyterhub 5 and focus on getting it out.