Housepets search engine!

Now with new GALLERY

Moderators: Hagus, SeanWolf

Post Reply
User avatar
MasterAbsinthe
Posts: 204
Joined: Wed Jan 22, 2020 3:01 am

Housepets search engine!

Post by MasterAbsinthe »

Hello Housepets fans! Today I'm starting an ambitious project to create the Housepets Search Engine™, potentially a very powerful tool for anyone who would like to study Housepets!

Image

Let me introduce myself: I'm an editor over at the Housepets! Fandom Wiki. As you can imagine, writing about the comic requires alot of research into the different story arcs, characters, and so on. For each page that I write, I need to have a full picture of the subject in order to faithfully represent the comic. Unfortunately, it's not so easy to effectively navigate the comic to find specific comic strips or information on overarching plot points like the Cosmic Game for instance. I've had to make up for this by memorizing a lot of the comic and just being able to remember things, but I've always thought: "I wish there was a better way to search the Comic!!"

And I came up with the dream of the Housepets Search Engine™, a website like Google where you can search the Housepets comic for any comic strips you'd like: search by character appearance, search by comic title, search by publishing date, and so on.

And today I start making that dream come true! How do I do that I hear you say? I have recently gained a lot of knowledge in what's called "web scraping", which is the art of programming a bot that can go to a website and "scrape", or gather, the information inside, so that I can get huge amounts of data very fast. Basically it involves a lot of this:

Image

Yes... it can be a headache sometimes... but web scraping is a very powerful tool indeed.
My idea for the Housepets Search Engine™ is that I start by writing a bot that goes to the Housepets comic website and goes through ALL of the comic strips, every single one, and gathers all of the information possible from each comic: the title, publishing date, which characters appear, and which story arc the strip is from (and more!), all this information can be put together in a HUGE database, and essentially this can be used as a look-up table to search the comic. It would look something like this:

Image

With this Housepets Database, specific searches can be done. For instance, it's possible to search for every comic strip that includes two characters, in order to see how the two characters have interacted throughout the comic. If you remember part of a comic strip title, you can search by that and easily find it! And even better, I'm also planning on using a text-recognition algorithm so that I can make a transcript of the text inside the comic strips. This way, you can search the comic by what words the characters have used, expanding your ability to navigate the comic.

So yeah, very big ideas indeed! Now to actually make it though... The first step is, as mentioned, making a complete Housepets Database. This would take the shape of a table like the one in the example above. The next step would be to make a website or application that any Housepets! fan can access and search with. This is the tricky step that I'm not sure how to solve... but I'm sure there is a solution. In any case, stay tuned for the progress!
Last edited by MasterAbsinthe on Thu Sep 16, 2021 1:44 pm, edited 2 times in total.
I specialize in wolf head avatars.

User avatar
Harry Johnathan
Posts: 1292
Joined: Thu Nov 12, 2020 12:10 pm
Contact:

Re: Housepets search engine!

Post by Harry Johnathan »

It seems like a lot of headache for little results.
It seems like a lot of headache for little results.
41892193d0fe7652ff8a415ce6a9afb9.jpg (86.7 KiB) Viewed 266 times
Pray for those who have fallen away, to be brought back to God. To their loss they are crucifying the Son all over again and mocking him.
Hebrews 6:6-7 (Paraphrase)

User avatar
Hagus
Moderator
Posts: 414
Joined: Sat Aug 24, 2019 9:43 pm
Contact:

Re: Housepets search engine!

Post by Hagus »

Personally I think it's a really cool idea, especially for people who are looking for specific strips or characters and don't know the comic itself all too well. It does seem like a lot of work though, and I wish you luck on the development :D
Image

User avatar
MasterAbsinthe
Posts: 204
Joined: Wed Jan 22, 2020 3:01 am

Re: Housepets search engine!

Post by MasterAbsinthe »

Harry Johnathan wrote:
Wed Sep 15, 2021 11:36 am
It seems like a lot of headache for little results.
Well, I will admit that this is something that will mostly be beneficial for niche users like me, who need to search the comic regularly and specifically.

And additionally, think of it like shooting a rocket to Mars: it won't necessarily lead to great rewards, but the process of scientific discovery and development is huge!

Basically, I'll be pouring dozens of hours on a furry webcomic in the name of science.
Hagus wrote:
Thu Sep 16, 2021 7:48 am
Personally I think it's a really cool idea, especially for people who are looking for specific strips or characters and don't know the comic itself all too well. It does seem like a lot of work though, and I wish you luck on the development :D
Thank you!

Time for our first update!

I decided to start the project by solving the hardest part: making an algorithm that can "read" the comic strips and find the text inside the speech bubbles, and turning that into computer text (that you could search for). To do this, the algorithm needs to first edit the image to make it black-and-white, so that the text is as clear as possible, here's Custom Ink illustrating the point:

Image

Then, it can use a so-called Optical Character Recognition engine that does the actual "reading". After some tweaking, I found the best settings that seem to work well on most comic strips:

Image

The resulting text is on the left, and as you can see it's not a perfect job, but it seems to work well on most cases. Here's another example that works splendidly:

Image

I will do some more testing of this text-recognition, and then I will move on to actually making the algorithm that goes to the Housepets website and gets the information we need. Stay tuned!
I specialize in wolf head avatars.

User avatar
DDeer
Posts: 675
Joined: Mon Aug 17, 2020 11:36 am

Re: Housepets search engine!

Post by DDeer »

I'm sure some will find this useful, clever stuff Absinthe!!
Disproportionately Dappled

User avatar
Amazee Dayzee
Posts: 21344
Joined: Tue Aug 12, 2014 6:24 pm

Re: Housepets search engine!

Post by Amazee Dayzee »

I am liking what you are showing us so far! Keep up the good work!

Post Reply