upvote
I think its kinda cool, but I dont really get it? So I can gather and search my browser history + most websites i have visited?
reply
This sounds very interesting. Wanted something similar, so going to check it out:)

One thing at first look, like a little feature request ;).

Personally I think it would be very nice if it was possible to index different sites to different search indexs. So you could separate different stuff, like job, different projects you have, and other stuff and then last everything else etc to there on search index db.

So it would be possible to have like "profiles" you could easy choose from. So in the addon, when you click on the hister icon on the toolbar there would be a list of profiles that you could choose where to index the site to. But also a setting to choose which domains and sites that always should index to one search index profils. And all sites that dont is added to a profile is added to the default index instead.

For me this would make it much more easy to find stuff and sort out what Im looking for (for me things get very "thing"/project based what I want to find), so having this..

1 - The "profiles" would become like an important filter. When you search you could easy mark one or more "profiles" which you would search from, and you remove alot of unwanted data automatically, specially when you index every site you visit by automatically.

2 - It would also make it easy if the db gets big over time to remove data that is less important later on and that you dont need to be index anymore.

3 - It would also make it easier to backup only the most important search index DB:s if the DB for the differnt search index:s where in different folders/files. I guess the "default index" that index all sites could easy get big, but there it would be alot of not important data. So would be nice to be able to easy just use a backup program and only backup the only important search databases, to save space.

Thanks

reply
>Hister builds a personal search index from pages you visit, bookmarks, browser history, local files, and crawled websites.

Immediately interested and will check it out, thank you! I've wanted a "search stuff you've seen online" tool for a long time, but everything seems to be research-oriented or "archive but don't search" or some weird combination that means it's nigh useless to me. I've got decades of bookmarks and archives and I've kinda been stuck grepping them at best (it's rare but I do sometimes want a page I saw once three years ago and I love having that option), while hoping someone would build something better.

One question if ya don't mind, while I explore: any chance of singlefile support? Content-extraction is useful in lots of situations (e.g. wallabag) and it's a great default, but sometimes it fails and sometimes you really do want the page, relatively close to how it actually was. Singlefile does that much better than most, and it does so well enough (and manually-handle-able enough if needed) that I don't feel any desire to switch to WARCs or similar.

Though specifically I'm probably looking for something like "content-extract everything" + "key combo to save singlefile version too" + "upload singlefile archives to backfill / recover". Like 99% of the time content extraction is preferred, and I'm glad to see it... it's just not always enough, and having to go elsewhere for exceptions breaks a lot of the utility.

reply
Exactly! I had the very same issues before Hister.

> One question if ya don't mind, while I explore: any chance of singlefile support?

Yes, partially. Hister can already import HTML files created by SingleFile, but there is no direct integration yet. In the longer term, I would like the SingleFile extension to be able to send snapshots directly to Hister.

reply
I assume it content-extracts that on upload? I'd really like to move storage into hister too, if possible. That way you could also switch from an extracted view to a "full" view in the UI. Though I assume that'd be fairly simple to build later.

Overall I really like what I'm seeing, it ticks a lot of important boxes for me and it's pleasantly straightforward. Hopefully I'll find time to contribute!

reply
> I assume it content-extracts that on upload?

Hister always stores the original material.

> That way you could also switch from an extracted view to a "full" view in the UI.

It isn't even needed, we just need a SingleFile specific extractor (an interface in Hister to parse specific page content and provide custom previews) that provides the full original HTML for the preview panel.

> Hopefully I'll find time to contribute!

I'd appreciate it. <3

reply
Oooh, now I see the extractor-view setting in the UI. Yeah, that's essentially perfect \o/

Thank you again!

reply
I came here to mention https://github.com/gildas-lormeau/singlefile, I couldn't get it to do what I want so I built my own, for watched domains it pushes a copy of the serialized DOM to a local search database. For structured data, either extract and enrich in the browser or enrich on the server side.

Would Hister support this basic workflow? I'd love to retire my own software.

The next phase was going to move to a recording proxy.

reply
I was thinking about making a "personal data search engine". I have almost 30GB of email archives and a bunch of Google Drive and offline files. I am thinking about downloading it all from the cloud and storing it on a large RAID array, with an indexing and search interface (and possibly a local MCP server).

Would Hister be suitable for this? Can it index mbox files? Would it handle this amount of data? Does it have a search API so I can build an MCP server?

reply
> Would Hister be suitable for this?

The indexer it uses (Bleve) can handle millions of records according to their docs, but sure that Hister would be the best choice for this task. I'd probably use Meilisearch (https://www.meilisearch.com/).

> Can it index mbox files?

Not yet.

> Does it have a search API so I can build an MCP server?

It has both search API and MCP server endpoints.

reply
Do you plan on adding filterable tags to the ui? Rather than just metadata.
reply
Very cool. Does it work to connect my phone and laptop to the same firehose?

I often find myself irritated because I read an article on my phone 6 months ago and the history is gone.

reply
You can host Hister on a home server and access it from multiple devices.

Automatic page capture on mobile currently requires Firefox, since mobile Chrome does not support browser extensions.

reply
What’s the safari story look like right now?
reply
Safari support is currently stalled in this PR: https://github.com/asciimoo/hister/pull/46. I do not have access to macOS, so help with development and testing would be very welcome.
reply
What would be a typical size of the search index, let's say after 5 years of intense browsing?
reply
It depends on what you consider intense browsing. An indexed document uses about 100KB on average because Hister stores the full original HTML for offline previews. If storage is a concern, you can disable offline previews, reducing the average document size significantly.
reply
You should integrate with Karakeep.app, it's only natural that you'd want both a search engine, and a nice "archive" and "article pretty view" features :)
reply
What do you mean by integration? Hister already supports importing data from Karakeep, Linkding, Linkwarden, Readeck, Shaarli, and wallabag: https://hister.org/docs/import
reply
interesting you used AGPL 3 licensing, are you planning a hosted version?

would've been great with a more liberal license

reply
> are you planning a hosted version

Not in the near term. Right now I am focused on developing Hister rather than operating a hosted service. There are already plenty of centralized hosted search engines, so my longer term interest is in federation and distributed search. I want to make the core system mature first.

> would've been great with a more liberal license

It depends on how do you define liberal. =] I chose AGPLv3+ because I want Hister to remain free software and available to their users.

reply
AGPLv3 is awesome, great choice!
reply
it prevents certain innovations to be derived from it but with LLMs I do think these licensing are pretty much moot.

i still think it could've benefited by Apache 2.0 which more or less gets you to your goals

reply
it would not! asciimoo, thanks for using AGPL, I've been using hister for months now and it helped me a lot
reply
I've been using LinkDing + SingleFile for this. Nice to have another option!
reply
That is a great combo. My main friction with it is having to manually capture pages I want to keep.
reply
[dead]
reply