@criztovyl's forgefed web log


#1

Hello there,

I am motivated to work on ForgeFed but I am not really sure where I can help.

I know there is Vervis, but I would like to throw in the idea to maybe implement a simple, generic AP mock we can throw things into the inbox and send things to other outboxes, just for getting a feeling for AP. We then could extend the vocabulary step-by-step to simulate forgefed there.

Just thoughts, but I like the idea of the AP “mock” and will play around with that idea a little bit. :slight_smile:


Reference implementations and bridges
#2

Hi @criztovyl!

There are definitely various ways to help :slight_smile:

Soon I will start writing a draft explaining the extensions and vocabulary I’m using. Ideas for things you can do that don’t need the draft:

  • Talk to the devs of existing forges such as GitLab CE, Gogs, Gitea, Pagure etc. and see if they’re open to changing their system to support federation and remote collaboration; make a list with the status of each forge’s feedback on this and willingness to support federation
  • Pick forge software such as the ones above and start patching it to support remote users in the database. For example, in an issue’s comments, allow a comment to be authored by either a local user or a remote user from another server.
  • Pick forge software and look into supporting the ActivityPub basics in it, such as storing remote actors and their HTTP signature keys in the database, and producing and verifying the signatures. And support for parsing and encoding ActivityStreams JSON(-LD) documents.
  • Generally, implement some simple ActivityPub client/server if you like, which can be used for playing with ForgeFed. I guess especially a client, because there don’t seem to be many ActivityPub clients

Soon I’ll be ready to deploy a basic federation test/demo. When that’s ready:

  • Test it, play with it, report bugs and feedback
  • Test it against other servers
  • Write a client, test with the client, report results

Once I start working on the draft:

  • Write comments, feedback, ideas
  • Implement it in an existing federated server
  • Implement it in an existing forge
  • Implement a CLI/TUI/GUI/web client

#3

For tracking I will collect some links here :slight_smile:

GitLab: https://gitlab.com/gitlab-org/gitlab-ce/issues/4013
Gogs: https://github.com/gogs/gogs/issues/4437
Gitea: https://github.com/go-gitea/gitea/issues/1612
Prague:
https://pagure.io/pagure/issue/2335

While looking into building an AP mock, if found ActivityPHP and JSON-NS.

ActivityPHP unfortunately requires all the properties in the JSON-LD data to be explicitly defined and fails otherwise (e.g. Mastodon/Pleroma publicKey), that makes it rather hard poking around in the Fediverse. I would prefer to ignore unknown properties, will open an issue for that ^^ Otherwise ActivityPHP seems fine, especially having the basic AP and AS objects (Like OrderedCollection) already implemented is nice.
(Or I learn go and use go-fed.)

JSON-NS seems to be interesting regarding the fact that I think (and was already discussed previously in ForgeFed context, if I am not mistaken) that not every AP software should be required to fully understand JSON-LD in all it’s complexity.

So far for now, I will maybe try to summarize and collect interesting things from the issues. (ForgeFed is already mentioned in the issues, except for GitLab.)


#4

criztovyl -

im not sure exactly what you meant by a “mock”; but i think you are imagining something like the demo peers we are discussing in this thread: Reference implementations and bridges

as for ways to help, you could work on one of those, if you have experience with python, ruby, or golang - but there is one other thing that fr33domlover did not mention that i would put at the highest priority at this time; which is doing some CSS design for vervis - that is of course only superficial; but right now there is none - its just a MFWS - appearances are of little concern to implementers; but have a great impact on many people who would not see the beauty behind the plain face - so i would like to encourage anyone with design experience to help give vervis a make-over


#5

Hello Bill,

yes, that „mock” I refer to is kind of a demo peer, but without any real implementation besides the handling of AP and just for experimenting around with AP, non-specific to ForgeFed.

Regarding vervis design, I tried to set up a vervis environment locally last week and did not get to finish it, had to abort the initial stack build downloading and compiling half of the internet (:D) and did not continue yet.


#6

I am currently struggeling with setting up a local instance of vervis, to play a little bit around; where is the right place to ask for help in that case? :slight_smile:

My current issue is that I cannot run on http://localhost:3000/, vervis wants to run on https://localhost/ albeit being configured differently. I will try what happens if I configure https on port 443 and ignore cert errors in my browser. ^^


#7

Thank you for trying and reporting :slight_smile:

I saw you opened a ticket on the Vervis ticket tracker, that’s indeed the right place! In addition, you can ping me on IRC or XMPP.

I commented on the ticket, and I’ll look into it soon :slight_smile:


#8

I tried to use a combination of an /etc/hosts entry (vervis.localnet) with Port 443 and the instance name set in settings.yml; it works so far after setcap for 443, but it seems like vervis now responds plain http on port 443. i.e. curl http://vervis.localnet:443 responds (same 400 bad request due to hostname mismatch as before) and hangs for curl https://vervis.localnet at the ssl client hello (or handshake, not entirely sure right now).

So my question is how you run vervis, if not via stack exec vervis. ^^
Somehow you must test your changes, right? :smiley:

In other news I started looking into gogs/gitea regarding initial AP (well, rather AS) integration. They already have an API that responds JSON, I will look further if some AS things can be added similar to how that API is implemented. :slight_smile:
I also want to look into how one could integrate go-fed.
I decided to do this for gogs; gitea has, to me, strange contribution guidelines that forbid pseudonyms for contributions.

I also made some progress making up my mind regarding BITT federation.

As a final note, I will rename this thread as my worklog, it’s not “Where to help” anymore. ^^


#9

@criztovyl, thank you for the detailed report about Vervis :slight_smile: If you want to try the hosts file approach, put Vervis on port 80 (not 443) and browse it in plain http. If that fails, maybe you’ll need a reverse proxy for TLS.

But that would just be a workaround: This problem is a bug in Vervis, that I should definitely fix :slight_smile: I’ve been working around it by testing my work on an actual server, but it’s probably time for me to just fix the bug :slight_smile:


#10

I will setup a reverse proxy then, that was the missing link. Although it’s not the most convenient way to go, but it’s not too complicated either. Maybe that’s a hint worth adding to the INSTALL.md.

Not familar with darcs yet, what would be the best way to share my to-be-made changes with you(r repo)? :slight_smile:


#11

Aaand my local instance is up-and-running. \o/


#12

I’ve added 127.0.0.1 vervis.localnet to my etc/hosts and configured a reverse proxy in my local Apache as below.
Then ran vervis via env INSTANCE_HOST=vervis.localnet stack exec vervis and after clicking away the cert error for vervis.localnet I can access local vervis via Browser. :slight_smile:

<VirtualHost vervis.localnet:443>
        ProxyPass "/" "http://localhost:3000/"
        ProxyPassReverse "/" "http://localhost:3000"
        ProxyPreserveHost On

        GnuTLSEnable On
        #   A self-signed (snakeoil) certificate can be created by installing the ssl-cert package. See /usr/share/doc/apache2.2-common/README.Debian.gz for more info.
        GnuTLSCertificateFile   /etc/ssl/certs/ssl-cert-snakeoil.pem
        GnuTLSKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
        #   See http://www.outoforder.cc/projects/apache/mod_gnutls/docs/#GnuTLSPriorities
        GnuTLSPriorities NORMAL 
</VirtualHost>

#13

I am unable to find out how to make my (dummy) changes to the templates appear.
Another stack build + stack exec vervis does not help, that does not make my changes visible, only stack build --force-dirty works, but that’s… very… slow (rebuilding from scratch) and not very productive.
I’ve set development: true and corresponding values in settings.yml, but that does not seem to help either.

As much as I’d like to help out with vervis, this is no environment I can work in; sorry.
This might be related to me not being familiar with the stack/framework and Haskell.
What’s the framework you are using, @fr33domlover? Some hints would be very valuable, I am just randomly poking around in the dark right now.

I will focus on other ForgeFed-related tasks for the moment.


#14

Hi @criztovyl!

Thank you for making an effort to figure out these details! I’m sad to see how complicated it is to launch a development instance. It’s supposed to be much easier, I need to fix the setup.

Here’s the situation with rebuilds and templates:

  • The Haskell compiler has a mechanism for detecting changes in those template files. In practice it usually doesn’t detect them. I’m not sure why yet, and I’ve made several compiler and library upgrades since the last time I checked this problem. What I’ve been doing is using touch or an actual little edit on the Haskell file that includes the template. For example, for the templates/default-layout.hamlet, that file is src/Vervis/Foundation.hs. You can use grep -r to find which source file to touch. I know it’s annoying, sorry :-/
  • The good way to develop templates is to run a live instance that automatically gets updated, you can do that using stack build yesod-bin and then stack exec yesod devel, and the app then runs locally on port 3000. I haven’t used this in a long time and possibly it may fail, and I need to fix it.

As to submitting changes, Vervis doesn’t have a UI yet for comparing repos, so my suggestion is:

  • Create a Darcs repo on dev.angeley.es or at hub.darcs.net
  • Put your patches there
  • Ping me on IRC / here / dev.angeley.es ticket comment, and I’ll look at your patches

If you want to skip the Darcs stuff for now, you can paste the diff somewhere on the internet, along with a screenshot.

(Even with yesod devel, templates edits may not trigger a rebuild, but Idk, I haven’t used this in a while)


#15

Solved the templates not triggering rebuilds by adding it to extra-source-files in vervis.cabal (templates/*.hamlet). cf https://github.com/commercialhaskell/stack/issues/4360; my stack seems to be too old (looking at you, Debian xD), will look for a convenient way to obtain a newer version that does not involve piping curl into bash.

Will see what yesod devel says. You refer to it also in the README, but also as might-be-broken, but now I have stack build --file-watch as a fallback.


#16

After some (more) days of being busy with other things (or being lazy) I would like put here the use-case I have in mind when thinking about forgefed.

Some days back I was a comparatively active contributor to projects hosted on github, lurking around in the issues and helping with them. (This model of contributing seems to be called “triager”.)

The projects I was contributing to I found via my search engine when searching for existing software that could solve a problem I was working on. For example Tabula for extracting tables as CSV or even JSON from PDF files.

Sometimes I found and fixed bugs. Sometimes I suggested and implemented new features.

Back then my workflow was fully centered on my GitHub notification (github.com/notifications). I regularly (much too often sometimes) checked those, where I then found new issues to help with, new responses to previous requests (could you please attach a log file), etc.

So far a prosaic description of my use-case.
It’s also the reason why I am often thinking about how to represent things in the UI - in the end I would like to have myforge.net/notifications where I can check for events that happend in the projects I follow, directly jump to the issues (preserving the look-and-feel of my instance), reply to them, etc.
Preserving the look-and-feel for me here is an important point. I don’t think it makes sense to somehow communicate my personal preferences to a remote instance when browsing issues. I rather think my local instance should allow me to browse remote projects and issues. I might need to expand on why exactly I don’t want to just follow hyperlinks to remote instances.

The only challenge now is to check where to build this into. I would prefer not to need to implement my own forge for that.

But so far I did not find a ready Free forge (except nonfree github) that supports my use-case. (esp not GitLab, which makes this extra sad.) I guess this is not to be blamed on the forges - in the end, right now, github has enough network effect and projects to make my use-case work there.
GitLab also is a DevOps tool, following other projects is out-of-scope.
Gitea and Gogs might be interesting. I did not yet look if there are notifications there as I want them.

Anyway, this then is the thing I am currently trying to wrap my head around. :slight_smile:


#17

I recently bumped into Gitlab-Triage (by way of @eliotberriot who keeps amazing me as an organizer, who implemented it for #funkwhale at https://dev.funkwhale.audio/funkwhale/automation). Although this does not cover your concern exactly, it’s a step in the right direction to monitor situations and poke humans to look into them.

My dream setup would combine the very simple cgit interface with the DevOps power of Gitlab, the convience of ActivityPub to indeed keep you in your own known visual environment, and the conversational power of Discourse, because I think people work things out better when they take the time to discuss and address all concerns (as in rough consensus and running code and humming).

I would be happy to see you unfold your vision of a distributed code forge, maybe by pointing at what other forges do well, and what’s missing from them.