Open /wp-includes/default-widgets.php
Look for the following line (Line 768 in version 3.4.2)
$title = "<a class='rsswidget' href='$url' title='" . esc_attr__( 'Syndicate this content' ) ."'><img style='border:0' width='14' height='14' src='$icon' alt='RSS' /></a> <a class='rsswidget' href='$link' title='$desc'>$title</a>";
remove
<img style='border:0' width='14' height='14' src='$icon' alt='RSS' />
Upload and test