Reference implementations and bridges


#1

it is clear that vervis will be the full-featured reference forge; but in order to expose compatibility bugs and to offer a reference for adoption into other forges, it will be valuable to write a few services in other languages - people are already asking questions to that concern and offering suggestions on how these should be implemented, most recently the OP of this forum thread: Where to help with ForgeFed and this notabug repo: https://notabug.org/zPlus/mcfi/src/master/docs/developers.md; so i figured we should start a discussion thread so as not to duplicate efforts - this OP is of course, not exhaustive and possibly inaccurate, but is intended as a seed for discussion of this topic

there are multiple way of accomplishing the general goal stated above - the first reference implementations already exists as a python service, and the beginnings of a ruby service, both currently on the ‘python’ branch of the notabug forgefed repo, using the approach of a headless demo dev service of which the interface code could be integrated almost verbatim, directly into the into existing forges without requiring any additional services - the python/flask code for pagure, and the ruby classes for gitlab, for examples - an analogous golang implementation would be needed for gogs and gittea using that approach - that is the most efficient approach of course, but is naturally the most intrusive and assumes the co-operation of the upstream forge devs in implementing or accepting patches for forge-fed compatibility upstream, or interested users running a forge-fed compatible fork if the upstreams are not interested in being forge-fed compatible - we do have reason to believe that the most popular self-hosted forges are interested though, so this is the approach i chose

the approach of the ‘mcfi’ implementation, as i understand it, is to have a dedicated forge-agnostic service mediating activity-pub messages for the forge; essentially translating them into standard GETs and POSTs that each forge will need to define and implement the internal handling for - it removes the need for existing forges to manage the activity-pub layer; but still requires the patching the upstream forge, in order to be useful in any way to the user - this approach is an unfortunate shim really, due to the fact that there are not many decent activity-pub libraries available

there is another potential approach of an API-bridge; which would also require the host to run a dedicated service, but would require the least (perhaps nothing) of the existing forge devs beyond providing a complete API for the forge - the downside to that approach is that currently, not all forges have a complete API, and that approach would naturally not support many features such as including remote forks and subscribers in the existing presentation indicators of the website, because most existing forges do not support actions from foreign users that would modify its database - pagure, may be the exception there; in that, it already supports merge-requests from foreign hosts running another instance of pagure - although, im not sure if or how that user is represented in the pagure back-end - it may be represented only in the merge-requests git repo

in any case, no existing forge is likely to treat activity-pub objects as canonical for any purpose; so in order to be actually useful to users beyond the simplest forking and perhaps disjunct event subscriptions, all of these approaches will require the upstream forge to honor requests from unregistered users and either to add their identity to their database as non-login “phantom” users, or attributing all of their activities to some generic “a-fediverse-user” - lets call that “fediverse-friendliness”

whether those request are handled directly in the original forge-fed form of activity-pub (the initial implementation), or as translated activity-pub messages via the existing forge API or to be otherwise handled internally by the forge) is a less interesting detail - where these approaches vary most significantly is in the level of intrusiveness into the existing forge code-bases

lastly, in order to implement the complete forge-fed spec (making that interface maximally useful), the original activity-pub objects also would need to be stored somewhere by the forge to allow complete propagating “linked” activity-pub behavior, such as threading future issue replies back into the activity-pub context - again, how that is handled depends on the desired level of intrusiveness into the forge code-bases, and the available activity-pub libraries that could be used as shims


Where to help with ForgeFed
#2

Hi, I’m zPlus. My repository was linked in the original post above.
I agree with Bill’s writeup above, but I would not call one approach more efficient than another. At the end of the day they are different ways to tackle the same problem, and whether developers prefer an “embedded” AP server/library or an “external” server is going to be a matter of specific requirements or maybe just preference. I don’t think they are “competing”, both have pros and cons. It’s almost like discussing if an embedded database is better than a database server, or if an embedded smtp server is better than a standalone one. Both should be able to cooperate by the way, since both speak AP.

My interest in making MCFI is to have something simple and tangible at hand, that can be quickly integrated with existing forges so that we can start experimenting with the protocol on real forges. Having a separate server makes this easy, I think, because it requires only minimal code in order to get a prototype up and running. Bugs aside, I can start the MCFI server right now and with the addition of only 1 controller, Pagure and Gitea could already start exchanging messages. What they do with those messages is up to the single forges. Maybe some more logic can be brought into the server so that forges have even less to deal with; right now the only logic that is embedded in the server is followers, which means forges do not need to keep track of that (but they can, if they want to). Developing multiple libraries can certainly be done, and it will be done, but it’s time consuming and we’re going to wait even more years (until they’re ready) before we start seeing some basic federation working across different forges. Upstreams however don’t seem very interested in the details, they kind of look to me only interested in understanding what’s available and how to use it. So what this MCFI is supposed to do is to give them something easy to use. All they have to do really is send messages to the server if they want to share something. Long term I guess, if people begin using forgefed, somebody will notice that MCFI has become a bottleneck and replace it with another server or with an embedded library, but that is a nicer problem to have than right now that there is no federation all. If there is an overwhelming interest from forges to use a library I will simply stop doing any more work on MCFI and look at those libraries instead.


#3

ok good pitch Z

all three of the approaches that i noted are viable and there are perhaps more to consider - there are pros and cons to each - i figured that was a good reason to start a discussion to flesh out precisely what those pros and cons are; and to see who wanted to work on either of them

what i meant by full-integration being more efficient, was only that it does not require an additional server with an additional database (a pro), if the existing forge server accepted the activity-pub message directly and stored the necessary state in its own database (a con)

i suppose we could also discuss the possible clients, such as an email bridge or a CLI or desktop client