Desired Projects
Part I: Archival
Our site is about finding great writing about games and linking to that writing every week. Eventually the sites hosting that writing disappear; we would like the content of that writing to be preserved. We used Amber for a little bit to preserve our linked material ,but it was causing some severe performance issues and also wasn’t meeting all of our use cases. For example, some sites do not return a 404 after removing their gaming content (Such as Zam). Other sites we maintain private archives of because they went down before we started using Amber. Finally, we’re on nginix and Amber doesn’t support that.
Due to the performance issues, we have turned Amber off. We would like similar functionality that:
1) Automatically archives the contents of a link when we publish a post containing that link
2) When a user clicks an external link from our website, detects when a site is down or unreachable and offers to redirect the user to the archived text
Part II: Full Text Search
When someone searches e.g. “Assassin’s Creed”, the current wordpress search only looks in our blogposts for the text “Assassin’s Creed”. However we don’t write our posts to name the game any given link is about. Amber can put everything we archive into an S3 bucket; we would like to scrape the contents of that S3 bucket for search results and return the links that have those results (and the pages of our blog with those links). We currently maintain that S3 bucket today. We would like to expand our ability to return relevant search results.
1) Perform searches on the contents of linked articles.