A great use for reddit is the ability to search posts and opinions about any niche topic. Will that be possible with Lemmy as it grows? Will I be able to Google “instant rice Lemmy” and get a comprehensive tier list of each brand?

I imagine search engines will have trouble with all the different instances(?). EDIT: Especially with instances that don’t have Lemmy in their name, I don’t think search engines would return them for Lemmy searches?

  • marsara9@lemmy.world
    104·
    1 year ago

    So I’ve been working on a solution for this.

    As I see it Google and others are going to have a hard if not impossible time to incorporate the fediverse, and the fact that the same content can exist on multiple servers.

    So I’m working on a search engine specifically build, for Lemmy at least. Where it’ll take you to whatever your preferred instance is when tapping on a search result.

    I hope to have a MVP up and running in a few more days.

    • mookulator@lemmy.worldEnglish
      17·
      1 year ago

      Can’t emphasize enough how important this is for the growth of Lemmy. Many people I know only access Reddit through google searches.

      • marsara9@lemmy.worldEnglish
        111·
        1 year ago

        Yep and I’m one of them. Go look me up on Reddit and I think I have maybe 20 posts over the 14+ years I was on the site. …joined Lemmy and immediately got frustrated that I couldn’t find anything. So I figured I take a crack at it. Especially since I couldn’t see how Google would ever be able to link me to my instance. Let alone make it easy to search the entire fediverse without having to write out every possible site, with new ones popping up every day.

        • Aki@lemmy.worldEnglish
          1·
          1 year ago

          I wonder if it’s possible to have a sophisticated search engine similar to Google’s, with BERT and kNN or vice versa. It would be the closest thing to Google search but specifically for Lemmy posts.

    • QuinicV@lemmy.worldOP
      7·
      1 year ago

      Interesting. I hadn’t even thought about how the fact that instance1.[post] and instance2.[post@instance1] is essentially the same thing and how search engines would handle it. Interested in what you come up with!

      • marsara9@lemmy.world
        8·
        1 year ago

        Thanks. If you do some digging you can find the project on GitHub but note that it’s a work in progress still. The UI is lacking and it’s rough around the edges but it’s “working”. And I still need to do some optimizations on the crawler itself, etc…

        It’s also going to be completely self-hostable just like Lemmy, etc…

    • silly goose meekah@lemmy.world
      2·
      1 year ago

      IDK, isn’t it the same for reddit? It also encourages crossposting, so the same content is on there several times. Maybe I don’t understand the fediverse well enough yet, so please correct me if I’m wrong.

      • 1st@kbin.social
        1·
        1 year ago

        On reddit you may have the same post twice but the comments will be different. On Lemmy, you have the same post on every federated lemmy with the same comments on all of them. With the way google handles websites right now, if they started including Lemmy instances in their web, it end up having hundreds of the exact same result each hosted on a different lemmy instance.

        Edit for clarity: All lemmy sites share their data with each other unless they explicitly stop doing so (defederating). This is why I can respond to your comment even though I’m on kbin.social and you’re on lemmy.world

    • static@kbin.social
      1·
      1 year ago

      The mastodon crowd was verry anti on search engines, and killed projects like this.
      But yea, do it! I think the lemmy/kbin crowd would mostly like it

    • Doll_Tow_Jet-ski@kbin.social
      11·
      1 year ago

      That is great. Thanks for the initiative. Have you considered contacting the people at DuckDuckGo so that that search engine can access Lemmy/Kbin content?

    • PierreKanazawa@fedia.io
      1·
      1 year ago

      Tried it, pretty cool, though seems still depend on search engines’ indexing. The instance that I’m on seems not indexed.

      Also it’s interesting it uses intext to identify whether the results are from fediverse.

  • OsakaWilson@lemmy.world
    12·
    1 year ago

    Digg.com was the big thing with Reddit trailing. Digg began tweaking the experience toward a more profitable model. I had already come to Reddit when they went too far and there was a sudden enormous migration from Digg to Reddit. Digg went from being THE social media aggregator to being nothing in a matter of weeks.

    Reddit is more deeply rooted, so I think it will stick around, I’m cool if Reddit keeps those who are happy with corporate model busy so we can do our thing here.

  • krigo666@lemmy.world
    91·
    1 year ago

    I think it is preferable to ask other search engines like DuckDuckGo to index Lemmy info. Google is full of garbage.

  • ghariksforge@lemmy.world
    8·
    1 year ago

    Depends on Google. These tech companies don’t like new platforms, especially those competing with established ones like Reddit. You’ll see that Google often discriminates against Lemmy or Mastodon.

  • RIotingPacifist@lemmy.world
    72·
    1 year ago

    Respectfully: Fuck that.

    If you want to find the best instant rice recommendations on Lemmy, Lemmy should have a functional post search function, rather than me relying on a malevolent corporate entity like google to index all the content.

    Search has gone to shit as the Internet has embraced social media sites, an upside of this is that wikipedia+Lemmy+key word search, mayas accurate as asking Google Bard or bing, and they can be built on entirety open tech.

    • Dr. Moose@lemmy.world
      5·
      1 year ago

      Cool rage but you dismissing search indexing is kinda hilarious. It’s not going away and it’s what makes the web. Would you rather have 3 big websites instead of indexed web?

      • RIotingPacifist@lemmy.world
        0·
        1 year ago

        Would you rather have 3 big websites instead of indexed web?

        That’s what we already have, I’d you need to find stuff by doing site specific googles, both google & that site have failed.

        The web is dead, it’s been dead for a while, now is the time to build something new in it’s wake that rather than depending on closed source algorithms, indexing 3 big websites, we could just search the 3 big websites directly.

        • PlexSheep@feddit.de
          1·
          1 year ago

          I disagree. I’m not sure why you say that. I Google stuff as a job and it’s certainly not just the big 3 websites. I personally rely on selfhosted searxng.

  • static@kbin.social
    4·
    1 year ago

    Reddit did not start out as the thing to google, it’s 15+ years old, only in the last 5y I started prefixing my google searches with reddit.

    • BrerChicken @lemmy.world
      7·
      1 year ago

      I actually found Reddit by googling things. I had seen it 5 or 6 times over a few years, and eventually I just went to the main site. I might have even used Reddit in the search before I joined. Regardless, I had recognized that all the best answers for tricky problems that I had were coming from Reddit before I even joined 11 years ago.

    • Fondots@lemmy.world
      3·
      1 year ago

      Everyone’s experience on this will be different, but I personally started using reddit about 12 years or so ago largely because at that point a lot of my Google searches were already pointing me towards reddit. I wasn’t necessarily going to google specifically to find reddit results, but since that’s where I kept ending up i figured I might as well go straight to reddit. And since reddit’s search function is and has always been trash, i pretty much immediately started using Google to search reddit.

    • QuinicV@lemmy.worldOP
      1·
      1 year ago

      I realize Lemmy needs to get much bigger for that to happen. My question was more directed at how search engines would handle the fediverse. Though I see now that that wasn’t very clear.

        • silly goose meekah@lemmy.world
          1·
          1 year ago

          Google could prevent lemmy pages from showing up in the results for example.

          Or they could adapt the protocol, make their own slightly tweaked version of it and let it die, which apparently often also kills the original protocol due to newly introduced compatibility issues, etc.

          Not sure about the second part, I read about it here somewhere where they mentioned an example of that happening as well but I can’t find it anymore.

        • balls_expert@lemmy.blahaj.zone
          1·
          1 year ago

          Like xmpp

          Make a giant instance, get all the content there by pumping users and making cool shit, slowly customize your instance and extend the protocol with features so that ours become incompatible in annoying ways

          Add wikis, overhaul user profiles, achievements, posting to your own profile, games, whatever, then get tired of supporting the fediverse interface and shut off the API

          You instantly can’t read 95% of your subscribed /f/s

    • 2pt_perversion@lemmy.world
      3·
      1 year ago

      I wish there was a way to get an entire Reddit archive over here. Realistically I’m still going to have to search Reddit because it has 10+ years of answers to obscure questions.

      • CascadeDismayed@lemmy.world
        1·
        1 year ago

        Minds more intelligent than mine are probably already at work on these problems. I’ve seen multiple discussion of people saying they are designing and working on solutions. It may take some time to see results, though.

  • hunte@lemmy.world
    3·
    1 year ago

    It’s a difficult problem but not principly impossible. One potentially good thing about Meta being involved is that if the user base is there, I’m pretty sure Google with their resources and other big tech backing will find a way and incorporate it into their engine.

    • QuinicV@lemmy.worldOP
      21·
      1 year ago

      I imagine that would be quite inconvenient… Especially as Lemmy grows and has potentially many more instances.

      • subtext@lemmy.worldEnglish
        2·
        1 year ago

        I believe that DDG has a shorthand for site:Reddit (without the .com). If lemmy gets popular enough DDG may implement a similar shorthand that incorporates the fediverse without us having to use a massive string. Like if it gets big enough, we may not have to solve this problem because others will see the value in making it easy.

        That’s my hope at least.

  • qwamqwamqwam@sh.itjust.works
    2·
    1 year ago

    I have seen at least one user claim they got a result from lemmy when searching a question on google. YMMV though. Lemmy is a fraction of the size of reddit, it will take time for posts to reach the level that google starts indexing them specifically.

    • IMongoose@lemmy.world
      1·
      1 year ago

      I got one. The Google link brought me to the instance though and not the thread. I was able to find the thread though, so it kinda worked.

    • NebLem@lemmy.world
      1·
      1 year ago

      Use the exclusion keyword for your search provider. For example on google lemmy -kilmister -motorhead will get you only Lemmy software results by excluding pages with “kilmister” or “motorhead” in their contents.