Can we insert or update FieldSets using apex code?

This is indeed possible via the Salesforce Metadata API. While this is a Web Service it is available through Apex like any other Web Service. There is a prebuilt Apex wrapper with samples here. The FieldSet metadata type can be used with the ‘create’ and ‘update’ operations on this API.

As if by magic it appears someone has just commited a FieldSet example to the library. Note that the Metadata API is an aysnc API, you can poll its results via Batch Apex or apex:actionPoller. Examples of doing both are included in the Github repo and Readme.

Creating a FieldSet via Apex…

    // FieldSet     
    MetadataService.FieldSet fieldSet = new MetadataService.FieldSet();
    fieldSet.fullName = 'Test__c.MyFieldSet';
    fieldSet.label = 'My FieldSet';
    fieldSet.description = 'Used by my VF page';
    MetadataService.FieldSetItem myAvailableField = new MetadataService.FieldSetItem();
    myAvailableField.field = 'TestField__c';
    myAvailableField.isFieldManaged = true;
    myAvailableField.isRequired = true;
    fieldSet.availableFields = new List<MetadataService.FieldSetItem>();
    fieldSet.availableFields.add(myAvailableField);

    // Create
    MetadataService.AsyncResult[] results = service.create(new List<MetadataService.Metadata> { fieldSet });

Updating a FieldSet via Apex…

    // FieldSet     
    MetadataService.FieldSet fieldSet = new MetadataService.FieldSet();
    fieldSet.fullName = 'Test__c.MyFieldSet';
    fieldSet.label = 'My FieldSet';
    fieldSet.description = 'Used by my VF page';            
    MetadataService.FieldSetItem myAvailableField = new MetadataService.FieldSetItem();
    myAvailableField.field = 'TestField__c';
    myAvailableField.isFieldManaged = true;
    myAvailableField.isRequired = true;
    MetadataService.FieldSetItem myAvailableFieldNew = new MetadataService.FieldSetItem();
    myAvailableFieldNew.field = 'ExternalField__c';
    myAvailableFieldNew.isFieldManaged = true;
    myAvailableFieldNew.isRequired = true;
    fieldSet.availableFields = new List<MetadataService.FieldSetItem>();
    fieldSet.availableFields.add(myAvailableField);
    fieldSet.availableFields.add(myAvailableFieldNew);

    // Update
    List<MetadataService.UpdateMetadata> updateMetadataList = new List<MetadataService.UpdateMetadata>();
    updateMetadataList.add(new MetadataService.UpdateMetadata());
    updateMetadataList[0].currentName = 'Test__c.MyFieldSet';
    updateMetadataList[0].metadata = fieldSet;
    MetadataService.AsyncResult[] results = service.updateMetadata(updateMetadataList);