Home 🞬 Podcasts 🞬 Crawl Budget Optimization for Publishers

Crawl Budget Optimization for Publishers

Vahe Arabian

May 21, 2021

Vahe Arabian

Founder and Editor in Chief of State of Digital Publishing. My vision is to provide digital publishing and media professionals a platform to collaborate and promote their efforts, my passion is to uncover talent and the latest trends for all to benefit.

GET THE INSIDE SCOOP ON PROVEN STRATEGIES FOR DIGITAL PUBLISHERS

Podcasts

Website Migration and Redesigns with Ben May

Podcasts

Google News Initiative Startups Playbook With Irene McKisson and Alicia Benjamin

Podcasts

Gutenberg for Publishers With Jeremy Fremont

Podcasts

The State of Contextual Video Advertising With Kai Henniges

See how one digital publisher increased its organic traffic by 600% with Google News Top Stories Carousel + best practices and troubleshooting tips

Latest Membership Insights

Saving Seconds: A Writer’s Guide to Mouseless Editing

The greatest time-saver I’ve found for editing is a little-known technique: mouse-less

Website Migration and Redesigns with Ben May

Ben May, CEO of The Code Company, talks to your host Vahe

Short-form Video and Web Stories Office Hours

Office Hours Join us as we discuss short-form videos, Meta’s plans to

Saving Seconds: A Writer’s Guide to Mouseless Editing

Website Migration and Redesigns with Ben May

Short-form Video and Web Stories Office Hours

How to Use Google Trends as a Publisher

LISTEN TO THE LATEST EPISODE

EDITOR’S PICK

State of Digital Publishing is creating a new publication and community for digital media and publishing professionals, in new media and technology.

We’re here to help industry members grow their audiences and forge successful media models together as peers.

We provide the resources, perspectives, collaboration, and news you need – so you get time back to work on what really matters.

Subscribe to receive a recap of our top stories in digital media & publishing content.

#clickok:active #elementor-popup-modal-10501{display:none;}@-webkit-keyframes bgslide{from{background-position-x:0;}to{background-position-x:-200%;}}@keyframes bgslide{from{background-position-x:0;}to{background-position-x:-200%;}}.wpp-widget-placeholder,.wpp-widget-block-placeholder{margin:0 auto;width:60px;height:3px;background:#dd3737;background:linear-gradient(90deg,#dd3737 0%,#571313 10%,#dd3737 100%);background-size:200% auto;border-radius:3px;-webkit-animation:bgslide 1s infinite linear;animation:bgslide 1s infinite linear;}@media only screen and (min-width: 768px){.cb-entry-content a:hover{text-decoration:none !important;}}.widget_bibblio_recent_posts p{border-bottom:5px solid #161616;display:inline-block;color:#161616;font-size:24px;text-transform:uppercase;padding-bottom:10px;font-weight:700;letter-spacing:2px;margin-bottom:40px;}.bibblio-branding{clear:both;color:#a4a4a4;font-size:small;text-align:center;margin:30px 0 2em;border-top:1px dotted #ccc;line-height:1.3;}.bibblio-branding span{padding:0 15px;display:inline-block;position:relative;top:-17px;background-color:#fff;line-height:1;}.bibblio-branding span a img{vertical-align:bottom;margin-top:5px;margin-left:4px;width:60px;}.vc_grid-container.vc_basic_grid .vc_grid-filter-dropdown.vc_grid-filter-left .vc_grid-styled-select{height:auto;}.vc_grid-container.vc_basic_grid .vc_grid-filter-dropdown.vc_grid-filter-left select{height:auto;padding:10px;font-size:20px;}.vc_grid-container.vc_basic_grid .vc_grid-filter-dropdown.vc_grid-filter-left i.vc_arrow-icon-navicon{display:inline-block;margin-bottom:10px;color:#000;}.member-content-tag-grid .svc_filter_main_div div:nth-child(3){display:none !important;}.member-content-tag-grid .svc_filter_main_div div:nth-child(2) select{width:60%;}#cb-nav-bar .cb-main-nav #cb-nav-logo,#cb-nav-bar .cb-main-nav #cb-nav-logo img{width:110px;}#cb-nav-bar .cb-main-nav #cb-nav-logo{visibility:visible;filter:progid:DXImageTransform.Microsoft.Alpha(enabled=false);opacity:1;-moz-transform:translate3d(0,0,0);-ms-transform:translate3d(0,0,0);-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0);margin:0 20px 0 12px;}#cb-nav-bar .cb-mega-menu-columns .cb-sub-menu > li > a{color:#f2c231;}.cb-review-box .cb-bar .cb-overlay span{background:#f9db32;}.cb-review-box i{color:#f9db32;}.cb-meta-style-1 .cb-article-meta{background:rgba(0,0,0,.25);}@media only screen and (min-width: 768px){.cb-module-block .cb-meta-style-1:hover .cb-article-meta{background:rgba(0,0,0,.75);}.cb-module-block .cb-meta-style-2:hover img,.cb-module-block .cb-meta-style-4:hover img,.cb-grid-x .cb-grid-feature:hover img,.cb-grid-x .cb-grid-feature:hover .cb-fi-cover,.cb-slider li:hover img{opacity:.25;}}@media only screen and (min-width: 1020px){}#respond,.cb-font-body{font-family:"Lora","Serif";}h2,h5,h6,.cb-font-header,#bbp-user-navigation,.cb-byline{font-family:"Arial","sans-serif";}.mc4wp-checkbox-wp-registration-form{clear:both;display:block;position:static;width:auto;}.mc4wp-checkbox-wp-registration-form input{float:none;width:auto;position:static;margin:0 6px 0 0;padding:0;vertical-align:middle;display:inline-block !important;max-width:21px;-webkit-appearance:checkbox;}.mc4wp-checkbox-wp-registration-form label{float:none;display:block;cursor:pointer;width:auto;position:static;margin:0 0 16px 0;}.cb-itemprop a{color:#3cbbb1;}.wpsm_panel{border-radius:50px !important;box-shadow:0px 0px 9px rgba(0,0,0,.17) !important;margin:10px !important;}ol{list-style-position:outside !important;padding-left:20px !important;}#yikes-easy-mc-form-1-EMAIL::placeholder{color:#555;font-size:14px;padding-left:10px;}.linguise_switcher span.linguise_language_icon,#linguise_popup li span.linguise_flags{width:15px;height:15px;}.linguise_lang_name{color:#fff !important;}.linguise_lang_name:hover{color:#fff !important;}.linguise_switcher span.linguise_language_icon{box-shadow:0px 0px 0px 0px #bfbfbf;}.linguise_switcher span.linguise_language_icon:hover{box-shadow:0px 0px 0px 0px #bfbfbf;}.linguise_switcher span.linguise_language_icon,#linguise_popup li span.linguise_flags{width:15px;height:15px;}.linguise_lang_name{color:#fff !important;}.linguise_lang_name:hover{color:#fff !important;}.linguise_switcher span.linguise_language_icon{box-shadow:0px 0px 0px 0px #bfbfbf;}.linguise_switcher span.linguise_language_icon:hover{box-shadow:0px 0px 0px 0px #bfbfbf;}.linguise_switcher span.linguise_language_icon,#linguise_popup li span.linguise_flags{width:15px;height:15px;}.linguise_lang_name{color:#fff !important;}.linguise_lang_name:hover{color:#fff !important;}.linguise_switcher span.linguise_language_icon{box-shadow:0px 0px 0px 0px #bfbfbf;}.linguise_switcher span.linguise_language_icon:hover{box-shadow:0px 0px 0px 0px #bfbfbf;}