First off, re: open sourcing Indieweb.xyz—I’m driving toward that. I’m in a private repo on Github right now. But, man. It’s unnerving to open that kind of code when… it’s running live on my server. So I am trying to find the security holes before releasing it.

I don’t have big plans for Indieweb.xyz, but one thing I’m planning on adding is a way to create a whitelisted sub. You basically can make a list of URLs and those are the only URLs that can submit to the sub. Who knows, I might use Vouch for this. I just want to use something that makes it effortless.

I wonder if this might be useful for quick collaboration. Name the sub, link a bunch of websites together and then go to town, sharing stuff.

I also am creating a few themes for people who want to run their own Indieweb.xyz as well, since the one I’ve got is designed for the web at large—clearly not the arXiv crowd.

Cool ideas!

A more advanced solution also scales. Which I would argue is a bad thing!

A large super-crawler ends up taking on the Whole Web. Which leads to a massive directory. Which leads to the big players having the resources to game that directory. Drowning out the individuals again.

If there are 100,000 bloggers, then we need 100,000 blogrolls.

This is the magic of it: Google can’t compete with 100,000 blogrolls. Yes, they can aggregate them. Destroying them in the process.

So, yes, we agree on how decentralized can really give us progress. And I agree that our problems are mostly not technological, but are human problems.

(This is similar to the discussion on decentralization out there that seems to think the answer can simply be solved by new technologies. Dat and IPFS are fantastic, but we can decentralize in vastly more meaningful ways today—but we don’t. We could learn to be better human discovery engines.)

If people began to link again, to read again, to explore again. Less statistics, fewer algorithms. More curation, more editors.

Great stuff today, Brad!

Mostly, similar to what coldbrain has said, I find blogs when they are casually mentioned on a blog or comment somewhere. Stuff like blogrolls and directories and such just don’t seem to exist. I know, because I’m constantly looking for them!

Now, these things do exist in the small enclave of the Indieweb. There is the Indieweb wiki, which has links all over it. And IndieNews, Indiemap, Blog Snoop and so on. But if I’m looking for blogs and websites that are out there—it’s impossible.

If I’m looking for a specific topic, I’ll Google “quilting blog” or I’ll look on Pinboard under the tag “chess”—and see what blogs come up.

But more often than not, I really want to read someone interesting. Someone’s stories and thoughts. To find all the great writers of our time that are out there. (Most writers I know that write in the literary tradition are lost as to where they should find readers now. It’s terribly ironic when you consider all the reading that is done on the Web in this age.)

This all excites me, though! It seems that there is still a frontier on the web. There is still a chasm to cross between all of us. We have a long way to go.

And I think that’s what drew me to the Indieweb. The answer will start here, in this group, because we’re thinking about it. I think about when Ward Cunningham came out with the wiki—it seemed like such a small, pointless invention. But what a masterstroke! What will be next?

Oh and one more thought about directories: I have a theory that exploring a directory is possibly not directly the best way to discover new things. They can be big and dry and tough to get through. I think they more directly benefit the builder of the directory and, also, those listed in the directory.

The builder of the directory explores and unearths other folks. They then leak into this person’s life in a myriad of ways. (For example, I began by simply linking to you once, in pursuit of new things, but now I follow you very closely.) The initial link begets more. Knotty, twisty—here I think of Sam Ruby’s word intertwingly—vines of links around each other.

And the recipient of the link is possibly motivated to build their own directory, so as to establish (or at least to not forget) their new network. So it can be viral. Blogrolls very much experienced this.

I think the idea behind Blog Snoop is solid—I mean you’re just talking about trying to define the edges of a certain community. I’m sufficiently convinced now (between Reddit wikis and ‘awesome lists’) that directories still serve this purpose. Find The Others.

I guess part of the problem is—what is the community? Bloggers? The Indieweb? The subset of the Indieweb that wants to talk about discovery? (Search, directories, blogrolls, etc.) I think you are shooting for larger than the Indieweb—bloggers, in general, right? That operate independently? So, do Medium blogs count?

Ok, so, the usability of the directory is central. This makes sense: a directory is a practical instrument. It needs to be elegant and tight. Using a blog as a directory is very novel and very convenient—and it can work! But I think the directory itself needs to be incredibly sweet to use: full of great stuff, well-organized and fun to use, if possible. I think you have to really want to visit the directory regularly.

Google won by distilling everything down to one box. It was actually fun to use Google because you could start typing and it would try to finish your sentence for you. Which I actually think more people enjoyed for its novelty than its usefulness. And it was useful, too!

So a few starter suggestions:

• The whole tag cloud is off to the side, as if it’s not important.
• I can’t see some of the words in the tag cloud, they are too small.
• The biggest words in the tag cloud are words like “General” and “Internet” which are almost non-categories because they apply to all blogs.
• It’s not clear how to submit my own blog unless I dig. People should really be encouraged to participate.
• The text is large, so lots of scrolling is involved. I think this is what the ‘awesome lists’ are doing right. It’s also what Chris Aldrich is doing so well with his blogroll.
• A dense list, like the one Chris has, also feels more active, for some reason.
• The thing you are doing perfectly, though, is the care in the descriptions. This is actually the most important thing once the directory is usable—and you have that already.

I am working on a personal directory right now, so my attention is there. But maybe if we keep talking about this, we can figure it out. Don’t give up—just keep talking and refining.

You’ve actually given me a great idea (I think it might be ‘great’, who knows) for Indieweb.xyz. I think I’m going to make a directory of the sites that submit to it. And it will also show the sub (‘tag’) that they most commonly submit to. It would be a simple change and might help me gradually collect links to blogs that I can go through over time.

Good luck, Brad! These tiny efforts may seem small in the face of massive social empires out there, but I think there are many people who are (or will) participate if they can just be found.

Ok, here we go. Hello my friend.

It is a real surprise to get a note from you. Frankly, I’m blown away by what you’re doing—and I felt that you were deeply embedded somewhere in your wiki, not exactly reaching out to pass notes. But I’m so glad.

The representation you’ve chosen for yourself is no illusion. If all I see of someone is pictures and quips, the image of that person is just a bit of shattered glass. But one thousand words cannot lie. They may be untrue, but they divulge so many more triangles to build that person with.

I am reluctant to say that even a verbose persona can ever be accurate, though. We start to believe that these pixels are us. I think they are just a good show. We are always in public here. And our souls know it.

I do look forward to the good show you have going. (Likewise: to you out there reading—hark, reader!—we are looking for you. Please read h0p3’s Find the Others as proof.)

I like what you are doing; my quest differs. I’m not looking for an ennobled species of thinker. I don’t want to avoid the unsavories. I am looking for Anyone. This is not right—there are plenty of Anyone’s around, aren’t there? Well, I am wanting to fall into fate, rather than to find only my most meaningful bosom friend. I want to travel down the river and stop at all the places that happen to be there along the way. And maybe a friend will crop up. Strange things, images and tales, too.

I am sold on it myself. I know I can communicate who I am, what I’m thinking, etc. with very low friction on my own medium-turf. My message integrates well into my identity when I use my own tools.

Yes, well, this is quite stunning actually. There is a sense on your wiki of a full use of hypertext that I haven’t seen before. It is the complexity of your organization and nomenclature. It is somehow still inviting. You have made a journal that is a vast puzzle.

I have been working, privately, on a directory of the web from a certain vantage. (I don’t know that my role is to journal; more, to explore out there.) And I think the real key that has been missing—from directories, wikis and engines—is artistry. To make it beautiful and elegant. Your work is a brilliant stone in that respect. Anyway, (hark, reader,) I’ll stop there.

Well, this will be fruitful. Feel no pressure to find some technology to make a connection with me. Climb down. We can link just fine.

Well, I think this is why I really like Webmentions as an expression of intention. In order to send one, you need to have your own blog. And you need to link back to the post. (AND, technically, you need your post laid out with microformats!)

This acts as a kind of wall around the garden. It acts like a gate. If you’ve got the code, you can get in. But likely you’re going to move on. There are just so many other places to harass and cause mischief on the web today. (Contrast with the massive silo gardens—if you are a member, you have all access to everyone there.)

I think a great benefit of social media could be that it provides an outlet for the masses so that careful enclaves can still be formed on the open web. I mean, look, there will always be bad actors in a group—groups seem to have a half-life where they grow too big and destabilize, the birth of a whole new era of drama and rage.

As for discovery, I think every blog needs a list of links to other blogs in their community. It’s like your friend list, but can be so much better than that. I want to read what you’re thinking, but I also want to know where else the discussion is happening.

Webrings are opaque; I can’t see where I’m headed. But a link list (blogroll, whatever) is like a gift.

Ok, yeah—this will be essential. I wonder if I can use the page’s title tag for this? Like: save the title tag the first time I check the site. Because it’s much less rare for the title tag to change than the body text.

I like the idea of the webring code. I just don’t have any influence on some of the sites that I am linking to. But yeah: mixing webrings and Indieweb.xyz is interesting.

As for the size of a directory (or webring), that is such a big problem on the Web. They need to have upper limits, for sure. If it gets too big, it feels (and does become) unusable. If it’s too large, then nothing in it is special. Like with these “awesome lists”—you are led to believe that the list is a severe abridgement, because the links in it are truly impressive.

With webrings, someone is special by virtue of discovering the ring first. So I can see closing admittance. I’m not that into webrings because it’s a pretty fragile link between all parties.

I’d like linking to take some effort, which also limits the amount of links you can have and makes them more potent.

Thanks for the ideas! Great stuff.

So, perhaps tags (or subreddit-style categories) are good for initial categorization and then a more detailed hierarchy is good for a competent editor. I also wonder: how did you track expired links? Changed links? Would you indicate that a story got an update?

Reddit has done a similar thing with wikis. By giving each subreddit a wiki, many are able to arrange a heirarchical directory of links. I guess I’m wondering if a wiki is a suitable replacement for a directory. Or if the only difference is having a crawler attached. (Which is a formidable difference.)

An idea I’ve had with Indieweb.xyz is to have users submit a finer-grained category using the u-category class. So they could submit:

<a href="https://indieweb.xyz/en/startrek" class="u-syndication u-category">
  xyz/startrek: Photos: DS9: Nog</a>

And it would place it in the permanent hierarchical directory (which crawls links to keep them fresh.) It feels like some moderation would be needed. But I am trying to stay away from that.

I appreciate your thoughtful replies. I am starting to both see how directories are present in our modern incarnation of the Web and desire some innovation for them.

Neat blog! It’s good to see so much longform blogging (like yours) still happening.

Well, yeah, I’m not sure how many bloggers like to spend time designing their blog. And if it’s all about the writing and communication for you—that makes sense! I don’t think it all needs to be organized and pretty, neither was the Old Web.

For me, I want to draw you to my blog where the post layout is preserved and you can feel like you’re visiting my little letterpress. To that end: thanks for saying hi!

So, with Horror Fiction, how do you decide how to rank things? PageRank clearly benefits all of the incumbents. I’m sure recency was important—people want to announce their new stories and articles and reviews. I’m sure people want to find lists of the most beloved stories. But I’m sure topic was important too: demonic possessions, monsters, mannequins. Although there might be spoilers in that kind of designation.

The ability to follow authors has got to be paramount in a community like that. So it would make sense for that kind of community to congregate in the Indieweb. Authors would post on blogs. (But don’t they usually post on wikis?) And then they ping the directory when they have new material.

Indieweb.xyz doesn’t have a crawler, but it does fetch the page and parse it and use its metadata. In a way, it works like a crawler where pages are submitted one by one. Even Reddit is a kind of crawler, performed by humans. So the crawler question is one of: how much do we need to go out and find random stuff unprompted?

Well, and the crawler would be useful for finding stuff outside the Indieweb. Unstructured, mostly undiscovered stuff. But the Indieweb’s insistence on structure has the benefit of weeding out spam. (For now.)

So, yes, I agree, directory + crawler. The next question is: have tags worked for discovery? (Good subreddits are just permanent, high-value hashtags.) Or can hierarchical directories still find a place?

Interestingly enough, I think two of the best hierarchical directories out there are eBay and Craigslist. And both are self-categorized! I wonder what they do to solve miscategorization.

Fantastic. Yes! This is exactly the problem. It’s very difficult for a new participant to make headway because they simply get lost in the pile.

I think all of these approaches need to be reconsidered just like you’re doing, but I want to focus on directories—which are still very much in vogue, but are unrecognizable when compared to the old Yahoo. The big directories these days are Reddit and Pinterest. (And Wikipedia, like you say.) Delicious was one of these too. They are directories because they are topic-based catalogs of thought.

Niche directories exist in the form of stuff like Pinboard, Hacker News, Lobste.rs and so on. So, Hacker News acts basically like a directory of thought for its community. And the users there spend their time pruning and curating this directory.

All of these directories struggle with a sort of memory failure—no one really plumbs the archives of these sites—but that makes perfect sense given that the focus is on absolute recency. Part of the spectacular failure of Yahoo-style directories was due to no sense of recency (a heartbeat) on all those links.

The nice thing about Yahoo was that you could categorize yourself. Reddit, Pinterest, Pinboard—you have to wait for someone else to find you.

To your point about filter blogs: I think there used to be an answer here. It used to be that for topic-based technology blogs, much of their grassroots content came from mailing lists. Mailing lists used to be the primary announcement system for software. (If you look at technology blogs, they are much more commercial now.) So the mailing lists acted like a completely open submission system where you could safely self-promote. And then blogs skimmed their favorite stuff out of these. (IRC and web forums also acted as support systems here.)

So here’s what I’d like to see in a directory:

1. Allows self-promotion. No one wants to leave a software (or fan art, essays, open question) announcement to their lonely blog. You want to push it out to the relevant community yourself.
2. Sorting isn’t driven by upvotes or algorithms. You shouldn’t need to have to figure out how to game the system. Yes, these algorithms help prevent spam. But they enable clickbait!
3. Drives users to the blogs themselves. Reddit isn’t just a directory—writers post their thoughts directly on Reddit. But those writers aren’t given all the tools. First off, the posts are limited in their layout. But also, they aren’t given syndication and identity tools like blogs have.
4. Personal views of the directory. Right now it seems that we have this idea that our moderators should have the power to decide who’s in and who’s out. But this was never true of mailing lists. I still like the idea of killfiles. You block the specific users and blogs that drive you nuts.

I am trying to accomplish some of this with my directory Indieweb.xyz, but I’m also not sure it will all work due to self-promotion having problems of its own—in addition to being a dirty word on its own.

At any rate, wonderful post. Thankyou!