File link appears when Chatter Feed is hidden and Page is refreshed

I have a requirement to hide the File Link in chatter feed. I have a VF Page and I am able to do this using the below code.

<script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script><script>               $j = $.noConflict();               $j(document).ready(function(){                    $j('#publisherAttachContentPost').hide();                 });</script><div id="chatter-container"><chatter:feedWithFollowers entityId="{!Object1__c.id}" showHeader ="true" oncomplete="refreshStatus()" reRender="statusPanel" /></div> 

But this works only when show Feed is clicked and the record is opened or refreshed. Below is the screenshot.

If the Show Feed is not clicked earlier

and if you open or refresh the record and click on Show Feed, then the File link still appears. Below is the screenshot.

Can some please tell me what am I missing.