Web scraping on Allociné DVD releases pages

Tools Webscraping by FlorentAlbert

During my entrepreneurial adventure (with the cultural agenda), I had to regularly update new events. A tedious task that could be partly automated. So, I had asked a quote for web scrape events published on cultural agendas.

Knowing that I was comfortable with HTML and CSS with some basics in Python, one of the developers told me that I should be able to « easily » scrape most agendas. But i have to confess, it was too new for me, my time was running out, and I had other tasks to do, so I gave up the idea (maybe too quickly).

A year has passed since and my progress in Python changed the game. What seemed too far to me at this time was now enjoyable. A lovely feeling!

Web scraping movies on Allociné

Regularly, I consult the last DVD releases on Allociné. So i wanted to list all these films with: title, press rating, spectators rating, actors, summary, url. 

This is a first webscraping exercise, and yes it was really fun to do!

0. Import BeautifulSoup library

# Import Librairies
import requests from bs4 
import BeautifulSoup
import pandas as pd
import numpy as np

1. Collect datas from the url

# create the list movie_cards, with as many nested lists as pages
pages = 25
url_param = 'http://www.allocine.fr'
fill_val = ''
movies_cards = []

for page in range(pages):
    url = f'http://www.allocine.fr/dvd/meilleurs/?page={page}'
    r = requests.get(url)
    allocine = BeautifulSoup(r.text, 'lxml')
    cards = allocine.find_all('div', class_='entity-card-list')
    movies_cards.append(cards)

2. Process datas

# fill our movie lists
movie_title = []
movie_url = []
movie_acteurs = []
movie_resume = []
movie_presse = []
movie_spectateurs = []
movie_amis = []

for pages in movies_cards:
    for card in pages:
        # add movies title
        movie_title.append(card.find('a', class_='meta-title-link', href=True).text)

        # add movies url
        movie_url.append(url_param + card.find('a', class_='meta-title-link', href=True)['href'])
        
        # add movies actors
        if 'meta-body-actor' in str(card):
            actors_blocs = card.find('div', class_='meta-body-actor').text.splitlines()
            actors_list = []
            
            for list_elements in actors_blocs:
                m = list_elements.strip()
                if m != '' and m != 'Avec':
                    actors_list.append(m)
            movie_acteurs.append(' '.join(actors_list))
            
        else:
            movie_acteurs.append(fill_val)
        
        # add movies resume
        if 'content-txt' in str(card):
            movie_resume.append(card.find('div', class_='content-txt').text.strip())
        else:
            movie_resume.append(fill_val)
        
        # add press, spectators and friends movie ratings
        if 'rating-item' in str(card):
            rating = card.find_all('div', class_='rating-item')
            i = 0
            tour = 1
            for elmt in rating:
                # we have to analyse each blocs separatly (cause same span/div class names)
                if tour%3 == 1:
                    if 'Presse' in str(rating[i]):
                        u = elmt.find_all('span', class_="stareval-note")[0].text
                        movie_presse.append(u)
                    else:
                        movie_presse.append(fill_val)
                        tour += 1
           
                if tour%3 == 2:
                    if 'Spectateur' in str(rating[i]):
                        v = elmt.find_all('span', class_="stareval-note")[0].text
                        movie_spectateurs.append(v)
                    else: 
                        movie_spectateurs.append(fill_val)
            
                if tour%3 == 0:
                    if 'Amis' in str(rating[i]):
                        v = elmt.find_all('span', class_="stareval-note")[0].text
                        movie_amis.append(v)
                    else:
                        movie_amis.append(fill_val)
            
                i += 1
                tour += 1
        else:
            movie_presse.append(fill_val)
            movie_spectateurs.append(fill_val)
            movie_amis.append(fill_val)

3. Group lists into a dataframe

# create our dataframe
allocine_recent_movies = pd.DataFrame(
    {'Title': movie_title,
     'Press Rating' : movie_presse,
     'Spectator Rating' : movie_spectateurs,
     'Actors': movie_acteurs,
     'Resume' : movie_resume,
     'Allociné URL' : movie_url
    })

# set full display on texts
pd.set_option('display.max_colwidth', -1)

# let's take a look
allocine_recent_movies.head()
Web scraping movies on Allocine

4. Export datas

# dataframe to HTML
allocine_recent_movies.to_html('.../190814_allocine_recent_movies.html')

# dataframe to CSV
allocine_recent_movies.to_csv('.../190814_allocine_recent_movies.csv')

Here is the HTML file (scraping done the 14th august 2019). 374 films, nice job BeautifulSoup!

- - -
Hope this post can be useful for you. Feel welcome to share your webscraping code in the comments!
- - -

105 comments

  1. antibioticsIsoge

    Special offer for antibiotics: can you buy antibiotics online and get discount for all purchased! Two free pills (Viagra or Cialis or Levitra) available with every order. No prescription required, free delivery.

  2. MichaelUnpax

    Antibiotics Online – Special offer: Save up to $498 – buy antibiotics online and get discount for all purchased!

  5. instructorships

    instructorships xyandanxvurulmus.1oiq5SQXHT0R

  7. bahis siteleri

    Güvenilir bahis siteleri arasında popüler ve Kazançlı bahis deneyimi için en iyi bahis sitelerini keşfedin. Güvenli ve eğlenceli bahis keyfi burada.

  8. bahis listesi

    Discover the best betting sites for a popular and profitable betting experience among reliable betting sites.

  10. BryanHeefs

    Thanks for magnificent info. What trips can you recommend in 2024? Astro tourism, eco diving, home swapping, train stations are the new food destinations,sports tourism, coolcationing, gig tripping, private group travel?

  14. fuck google

    bastan sona ta ta ta ananin amina kadar

  23. child porn

    thank you admin very nice blog

  27. gab

    The company has never elaborated on why this move was taken, although it did occur not long after Titan Poker threatened to pull out of the Canadian market. That may be completely coincidental, but it means that Skrill is no longer an option for Canadian online poker players or casino users. Not only does Skrill keep your bank information separate from online casinos and other sites, but it also functions with real time, instant transactions. That’s why many players opt for Skrill and other e-wallets when it comes to banking at online casinos. Why you should play at 4Crowns CasinoA £5,000 welcome bonus spread out across your first seven deposits, plus 450 extra spins!More than a thousand of the highest-quality games from the industry’s most fantastic studios.There is a good selection of quick and easy payment options.This is possibly one of the best UK online casinos.A crew dedicated to providing customer support is available around the clock.6. My Stake Top UK Online Casino for Casino + Bookie
    https://www.coloradosportsguys.com/category/poker/page/7/
    We love a top VIP program that treats high rollers well, but that’s not the only casino bonus to look out for. Our top high-roller casinos in the UK also have great welcome bonuses and reload bonuses to keep that bonus cash flowing. Although Boku is an excellent payment method, few Boku casino sites are available. However, you can be sure that the ones in our list of top deposit by Boku casino sites are high-quality and safe for UK players. High rollers and low rollers can enjoy all their favourite games without any worries like classic card games at Boku blackjack sites. Browse the entire Casino Guru casino database and see all casinos you can choose from. If you want to leave your options open, this is the right list of casinos for you. It might take you longer to find the best option though.

  29. vegas glass porn

    You can even sign up for a see-through version with Vegas Glass Kayaks.

  30. Laurene

    I am no longer certain where you’re getting your information,
    but good topic. I needs to spend a while learning much more or working out more.
    Thanks for magnificent info I was searching for this info
    for my mission.

  31. user-198966

    awesome

  34. pornky.com

    free hd pourn ggjgodherogg.x9oEBpVNuLy

  40. S'inscrire sur Binance

    Your point of view caught my eye and was very interesting. Thanks. I have a question for you.

  41. is gate io safe reddit

    Reading your article helped me a lot and I agree with you. But I still have some doubts, can you clarify for me? I’ll keep an eye out for your answers.

  44. milf ਪੋਰਨ

    ਗੇ ਪੋਰਨ madisonivysex.4tHUg9LWuBV

  45. Binance

    Can you be more specific about the content of your article? After reading it, I still have some doubts. Hope you can help me.

  50. Реферальный код binance

    I don’t think the title of your article matches the content lol. Just kidding, mainly because I had some doubts after reading the article.

  51. Lilibet bonus

    For de beste tipsene om Casino i Norge og spennende spillanmeldelser, bes�k kasinoekspert.com i dag!

  52. Casino Bonus Ved Registrering

    Topnorgecasino.com gir deg den mest omfattende oversikten over Norske Casino p� nett, slik at du kan velge de beste spillene og kasinoene for en uforglemmelig spillopplevelse.

  53. Norske Casino Bonuser

    Utforsk spennende Norge Casino-opplevelser og de nyeste spillene p� norgecasino.org for en fantastisk tid med online gambling

  54. Beste spilleautomater

    For de beste tilbudene og spillopplevelsene p� et Norsk Casino, bes�k betbonuser.com og utforsk et bredt utvalg av bonuser og spillalternativer.

  55. Lilibet Gameshow

    For den beste Lilibet opplevelsen i Norge, bes�k lilibetcasino.org og utforsk et bredt utvalg av spennende spill og tilbud.

  56. Lilibet Mobil

    For den beste Lilibet opplevelsen i Norge, bes�k lilibetcasino.org og utforsk et bredt utvalg av spennende spill og tilbud.

  57. Lilibet velkomstbonus

    For den ultimate Lilibet-opplevelsen i Norge, bes�k lilibetbonus.com og utforsk et bredt utvalg av spennende spill og eksklusive tilbud

  58. Lilibet Odds

    For den beste Lilibet-opplevelsen i Norge, bes�k lilibetnorge.com og utforsk et bredt utvalg av spennende spill og eksklusive tilbud.

  59. Casino Norge

    Utforsk spenningen av online gambling i Casino Norge med de beste anbefalingene fra bestecasinonorge.net

  60. Nye casino

    Utforsk spenningen av online gambling i Casino Norge med de beste anbefalingene fra bestecasinonorge.net

  61. Lilibet anmeldelse

    For de nyeste spillnyhetene og innsikten i casinoverdenen, sjekk ut Lilibet Blogg p� blog.lilibet.com/no/.

  62. Lilibet betalingsmetoder

    For an exciting Lilibet Casino experience, visit lilibet-casino.org and explore a wide range of thrilling games and exclusive offers

  63. Sultanbet Casino Willkommensbonus

    Bei Sultanbet k�nnen Sie eine Vielzahl von Spielen genie�en. Dar�ber hinaus finden Sie auf sultanbetcasino.net aktuelle Informationen f�r weitere Einblicke.

  64. Sultanbet Live Casino

    Bei Sultanbet k�nnen Sie eine Vielzahl von Spielen genie�en. Dar�ber hinaus finden Sie auf sultanbetcasino.net aktuelle Informationen f�r weitere Einblicke.

  65. Sultanbet Casino Willkommensbonus

    Bei Sultanbet k�nnen Sie eine vielf�ltige Auswahl an Spielen genie�en. Dar�ber hinaus bietet blog.sultanbet.com/de/ umfassende Informationen und Einblicke rund um das Casino.

  66. Sultanbet Live Casino

    Bei Sultanbet k�nnen Sie eine vielf�ltige Auswahl an Spielen genie�en. Dar�ber hinaus bietet blog.sultanbet.com/de/ umfassende Informationen und Einblicke rund um das Casino.

  67. Neue Online Casinos

    Sie k�nnen das beste Spielerlebnis im Online Casino Deutschland erleben. Au�erdem finden Sie auf onlinecasino-deutschland.net aktuelle Bonusangebote und Spieloptionen.

  68. Sportwetten Anbieter

    Im Online Casino Deutschland finden Sie unterhaltsame und gewinnbringende Spiele. Zudem k�nnen Sie auf neue-onlinecasinos.net die neuesten Online-Casino-Optionen entdecken.

  69. Online Casino Boni

    Bei Sportwetten Deutschland k�nnen Sie auf spannende Sportereignisse wetten und Ihre Gewinnchancen erh�hen. Dar�ber hinaus finden Sie auf beste-wettanbieter.net die besten Wettanbieter und lukrative Wettangebote.

  70. Sultanbet Sportwetten

    Bei Sportwetten Deutschland k�nnen Sie auf spannende Sportereignisse wetten und Ihre Gewinnchancen erh�hen. Dar�ber hinaus finden Sie auf beste-wettanbieter.net die besten Wettanbieter und lukrative Wettangebote.

  71. Sportwetten Anbieter

    Im Online Casino Deutschland k�nnen Sie eine breite Palette von Casinospielen erleben. Weiterhin bietet casinoonlinedeutschland.org umfassende Informationen �ber Online-Casinos und ihre Angebote.

  72. Deutsche Online Casinos

    Im Online Casino Deutschland k�nnen Sie eine breite Palette von Casinospielen erleben. Weiterhin bietet casinoonlinedeutschland.org umfassende Informationen �ber Online-Casinos und ihre Angebote.

  73. Sultanbet VIP

    Bei Sultanbet k�nnen Sie eine vielf�ltige Auswahl an Spielen erleben. Weiterhin finden Sie auf sultanbet-casino.com umfassende Informationen �ber das Casino und seine Dienstleistungen.

  74. Sultanbet Live-Wetten

    Bei Sultanbet k�nnen Sie eine vielf�ltige Auswahl an Spielen genie�en. Dar�ber hinaus bietet sultanbet-bonus.net umfassende Informationen �ber die Bonusangebote des Casinos.

  75. Canl� bahis

    Ma� esnas�nda bahis yapabilece�iniz ve oranlar�n anl�k olarak de�i�ti�i canl� bahis sayfas�nda siz de favori ma��n�z�n getirdi�i heyecana ortak olabilirsiniz.

  76. Bonus veren bahis siteleri

    Deneme bonusu veren siteler 2024 nas�l bulunur ve g�ncel deneme bonusu listelerine nas�l ula��l�r sorular�na sitemizde cevap bulabilirsiniz

  77. Sportotobet giri�

    Supertotobet ile �ans�n�z�n Mobildeki Adresi: Supertotobet.mobi, Kazanma Heyecan�n� Cebinizde Ya�ay�n

  78. Sportotobet g�ncel giri�

    Sportotobet ile Spor Bahislerinde Yeni Bir Boyut: Superotobetyenigiris.org ile An�nda Kazanmaya Ba�lay�n!

  79. Restbet canl� ma� izle

    Restbet ile Bahis Heyecan� Dorukta, Resbet1.org ile G�venli Giri� Yap�n ve Kazanmaya Ba�lay�n!

  80. Restbet g�ncel

    Restbet ile Bahis Heyecan� Dorukta, Resbet1.org ile G�venli Giri� Yap�n ve Kazanmaya Ba�lay�n!

  81. Sportotobet yeni giri�

    Sportotobet, spor bahisleri ve �evrimi�i casino oyunlar� sunan bir platformdur. Geni� bir spor bahisleri yelpazesi ve �e�itli liglerde y�ksek oranlarla bahis yapma imkan� sunar.

  82. Supertotobet giri�

    Supertotobet, �e�itli online bahis ve casino oyunlar� sunan g�venilir bir platformdur. Geni� spor bahisleri se�enekleri, canl� casino deneyimi ve pop�ler slot oyunlar� ile kullan�c�lar�na heyecan verici bir oyun ortam� sunar.

  83. Betpas

    Betpas, �e�itli spor bahisleri ve casino oyunlar� sunan bir online bahis platformudur. Kullan�c�lar�na geni� bir spor bahisleri se�ene�i sunman�n yan� s�ra, pop�ler casino oyunlar� ve canl� casino deneyimi de sunar.

  84. Betpas �yelik

    Betpas, spor bahisleri ve �evrimi�i casino oyunlar� sunan bir �evrimi�i bahis platformudur. Kullan�c�lar�na geni� bir spor bahisleri yelpazesi sunman�n yan� s�ra, pop�ler casino oyunlar� ve canl� krupiyelerle oynanan oyunlar da sunar.

  85. Betist com giri�

    Betist, �e�itli spor bahisleri ve �evrimi�i casino oyunlar� sunan bir �evrimi�i bahis platformudur. Geni� bir spor bahisleri yelpazesi sunarken, pop�ler casino oyunlar� ve canl� casino deneyimini de sa�lar.

  86. Mariobett

    Mariobet, �e�itli spor bahisleri ve �evrimi�i casino oyunlar� sunan bir �evrimi�i bahis platformudur. Kullan�c�lara geni� bir spor bahis se�ene�i sunman�n yan� s�ra pop�ler casino oyunlar� ve canl� casino deneyimi de sa�lar.

  87. Betist giri� 2024

    Betist, çeşitli spor bahisleri ve çevrimiçi casino oyunları sunan bir çevrimiçi bahis platformudur. Geniş bir spor bahisleri yelpazesi sunarken, popüler casino oyunları ve canlı casino deneyimini de sağlar. Yüksek oranlar ve çeşitli liglerde bahis yapma imkanı sunar.

  88. Tipobet365

    Tipobet, çeşitli spor bahisleri ve çevrimiçi casino oyunları sunan bir çevrimiçi bahis platformudur. Geniş bir spor bahisleri yelpazesi sunarken, aynı zamanda popüler casino oyunları ve canlı casino deneyimini de sağlar.

  89. Betpublic 2024

    Betpublic giriş sayfasına Twitter yani X üzerinden bu hesap üzerinden ulaşabilirsiniz. Twitter güncel, sorunsuz, hızlı ve güvenilir giriş adres burada yer.

  90. Betpublic

    Betpublic güncel giriş ekranına, bu sayfamız üzerinden betmatik giriş yapabilirsiniz. Yer alan link üzerinden güncel, yeni, sorunsuz ve hızlı giriş ekranına.

  91. binance Registrácia

    Your point of view caught my eye and was very interesting. Thanks. I have a question for you.

  92. smorter giremal

    hello there and thank you in your information – I have certainly picked up anything new from right here. I did then again experience several technical points the usage of this web site, as I experienced to reload the website a lot of times prior to I may just get it to load properly. I have been thinking about in case your web host is OK? Not that I am complaining, however slow loading circumstances occasions will sometimes impact your placement in google and can damage your quality score if advertising and ***********|advertising|advertising|advertising and *********** with Adwords. Anyway I am including this RSS to my email and could glance out for much more of your respective interesting content. Ensure that you update this again soon..

  93. smorter giremal

    I don’t ordinarily comment but I gotta tell appreciate it for the post on this great one : D.

  94. Deneme Bonusu Veren Siteler 2024

    Kaçak bahis siteleri yaptıkları promosyonlarla yeni üye elde etmeyi amaçlamaktadırlar. Türk sitelerinin en büyük özelliği sundukları deneme bonuslarıdır

  95. Bahis siteleri 2024

    Deneme bonusu ve güvenilir bahis siteleri ile bizimle en iyi oranlarla canlı bahis yapın. 2024’ün öncü deneme bonusu veren sitelerini keşfedin.

  98. 創建免費帳戶

    I don’t think the title of your article matches the content lol. Just kidding, mainly because I had some doubts after reading the article.

  99. OrenP

    I was looking through some of your blog posts on this site
    and I think this site is real informative! Keep posting.Money from blog

  101. owlpedia.org

    I’m extremely impressed with your writing skills
    and also with the layout on your weblog. Is this a
    paid theme or did you modify it yourself?

    Anyway keep up the excellent quality writing, it’s
    rare to see a great blog like this one nowadays.

  102. https://www.giantbomb.com/profile/ryalasdutv/

    Good day I am so thrilled I found your blog, I really found you by mistake, while I
    was searching on Yahoo for something else, Anyhow I am here now and would just like to say many thanks for a remarkable post and a all round entertaining blog
    (I also love the theme/design), I don’t have time to browse it all at the minute but I have
    book-marked it and also added in your RSS feeds, so when I have time I will be back to read a great deal more, Please
    do keep up the fantastic b.

  104. referencia de Binance

    Can you be more specific about the content of your article? After reading it, I still have some doubts. Hope you can help me.

Laisser un commentaire

Votre adresse de messagerie ne sera pas publiée.