Problem with funannote2 annotate parsing from funannotate-addons results

[OceaneMion]

Hi,
I saw that you Closed #90 as completed via #95,and thanks you for providing a correction, but I am still running into some problem with the parsing of funannotate2, even after downloading the new version.

I think the problem is mainly in the annote.py file:
if "ec_number" in fa:
n["ec_number"][i] = fa["ec_number"]
if "go_terms" in fa:
n["go_terms"][i] = fa["go_terms"]

Here you are taking ec_number, but the notation in the output of eggnog from funannote-addons is EC_number.
Same problem with go_terms from interproscan results it is written as go_term in the output file of interpro not go_terms, and for SignalP in the file it is written as SignalP: not SECRETED, I tried a correction that I do not know if perfectly correct:

cat > ~/fix_funannotate2.sh <<'EOF'
#!/bin/bash
S=$(python -c "import funannotate2,gfftk,os;print(os.path.dirname(funannotate2.file),os.path.dirname(gfftk.file))")
F=${S% }; G=${S# }
sed -i 's/n["ec_number"]/n["EC_number"]/;s/n["go_term"][i]/n["go_terms"][i]/;s/"ec_number" in fa/"EC_number" in fa/;s/fa["ec_number"]/fa["EC_number"]/;s/"go_terms" in fa/"go_term" in fa/;s/fa["go_terms"]/fa["go_term"]/' $F/annotate.py
sed -i 's/InterPro:") for s in v["db_xref"][i]):/InterPro:") for s in v["db_xref"][i] + v["note"][i]):/;s/s.startswith("SECRETED:")/s.startswith(("SECRETED:","SignalP:"))/' $G/stats.py
python -c "import funannotate2.annotate,gfftk.stats" && echo "patchs OK"
EOF
chmod +x ~/fix_funannotate2.sh && ~/fix_funannotate2.sh

But is seems to correct the problem?

"functional": {
  "go_terms": 5296, 
  "interproscan": 7840, ## Instead of 0
  "eggnog": 9449,
  "pfam": 7426,
  "cazyme": 265,
  "merops": 232,
  "busco": 3861,
  "secretion": 505 ## Instead of 0
}

}
}

Also I would like to ask you if you know why on some genes I have a very high number of different pathway and what could be a good way to determine which is not good prediction from the files output ?

Here is an example that I cut because it is very long:
Gd_00293-aad_split#contig_38:0-3546625 ANNEVO mRNA 25658 26938 . + . ID=Gd_00293-aad_split#contig_38:0-3546625-g3.t1;Parent=Gd_00293-aad_split#contig_38:0-3546625-g3;product=hypothetical protein;Dbxref=PFAM:PF13207.13,PFAM:PF12710.14,PFAM:PF00702.33;Note=Uracil phosphoribosyltransferase,COG:S,EggNog:ENOG502EAZH,InterPro:IPR027417 P-loop_NTPase,InterPro:IPR023214 HAD_sf,InterPro:IPR050582 HAD-like_SerB,InterPro:IPR036412 HAD-like_sf,G3DSA:3.40.50.300,PF12710 HAD,PF13207 AAA_17,G3DSA:3.40.50.1000,mobidb-lite disorder_prediction,PTHR43344 PHOSPHOSERINE PHOSPHATASE,SSF52540 P-loop containing nucleoside triphosphate hydrolases,SSF56784 HAD-like,Pathway:Reactome:R-MMU-2173791 TGF-beta receptor signaling in EMT (epithelial to mesenchymal transition),Pathway:Reactome:R-SCE-5689896 Ovarian tumor domain proteases,Pathway:Reactome:R-SSC-432040 Vasopressin regulates renal water homeostasis via Aquaporins,Pathway:Reactome:R-HSA-113418 Formation of the Early Elongation Complex,Pathway:Reactome:R-HSA-5673001 RAF/MAP kinase cascade,Pathway:Reactome:R-HSA-2565942 Regulation of PLK1 Activity at G2/M Transition,Pathway:Reactome:R-MMU-5620912 Anchoring of the basal body to the plasma membrane,Pathway:Reactome:R-CEL-171007 p38MAPK events,Pathway:Reactome:R-HSA-9948001 CASP4 inflammasome assembly,Pathway:Reactome:R-SCE-9696273 RND1 GTPase cycle,Pathway:Reactome:R-MMU-6791226 Major pathway of rRNA processing in the nucleolus and cytosol,Pathway:Reactome:R-HSA-2559586 DNA Damage/Telomere Stress Induced Senescence,Pathway:Reactome:R-HSA-888590 GABA synthesis,Pathway: release,Pathway: reuptake and degradation,Pathway:Reactome:R-GGA-5696400 Dual Incision in GG-NER,Pathway:Reactome:R-HSA-5693616 Presynaptic phase of homologous DNA pairing and strand exchange,Pathway:Reactome:R-HSA-9754706 Atorvastatin ADME,Pathway:Reactome:R-BTA-113418 Formation of the Early Elongation Complex,Pathway:Reactome:R-DME-4608870 Asymmetric localization of PCP proteins,Pathway:Reactome:R-GGA-1810476 RIP-mediated NFkB activation via ZBP1,Pathway:Reactome:R-DME-209461 Ubiquitination and degradation of phosphorylated ARM,Pathway:Reactome:R-RNO-111457 Release of apoptotic factors from the mitochondria,Pathway:Reactome:R-DME-983189 Kinesins,Pathway:Reactome:R-RNO-525793 Myogenesis

[OceaneMion]

Hi, sorry to bother you again,
I was just wondering if you’ve had a chance to look into the parsing issue with funannote2 annotate again ?

[aldendirks]

I'm having the same issue. I parsed interproscan, signalp, and antismash results with f2a. When I rerun the annotation command to incorporate them, they are recognized in annotation_misc but don't seem to integrated:

[Sep 17 09:02 PM] Existing Pfam-A results found, loaded annotations for 5119 gene models
[Sep 17 09:02 PM] Existing dbCAN results found, loaded annotations for 170 gene models
[Sep 17 09:02 PM] Existing UniProtKB/Swiss-Prot results found, loaded annotations for 1498 gene models
[Sep 17 09:02 PM] Existing MEROPS results found, loaded annotations for 208 gene models
[Sep 17 09:02 PM] Existing BUSCOlite results found, loaded annotations for 3254 gene models
[Sep 17 09:02 PM] Loaded 77 annotations from /scr1/users/dirksa/candida/seqs/funannotate2/ONCO0025/annotate_misc/antismash.annotations.txt
[Sep 17 09:02 PM] Loaded 5759 annotations from /scr1/users/dirksa/candida/seqs/funannotate2/ONCO0025/annotate_misc/iprscan.annotations.txt
[Sep 17 09:02 PM] Loaded 395 annotations from /scr1/users/dirksa/candida/seqs/funannotate2/ONCO0025/annotate_misc/signalp.annotations.txt
[Sep 17 09:02 PM] Found functional annotation for 6261 gene models
[Sep 17 09:02 PM] Annotation sources: {'db_xref': 6617, 'note': 9970, 'name': 1391, 'product': 1391}
[Sep 17 09:02 PM] Cleaning gene names and product descriptions using curated database
[Sep 17 09:02 PM] Found 38 new valid gene names/products that could be added to the curated database
[Sep 17 09:02 PM] See /scr1/users/dirksa/candida/seqs/funannotate2/ONCO0025/annotate_results/Gene2Products.new-valid.txt for details
[Sep 17 09:02 PM] Converting to GenBank format
[Sep 17 09:02 PM] Writing rest of the output annotation files
[Sep 17 09:03 PM] Annotation Summary:
[Sep 17 09:03 PM] 
{
  "genes": 6261,
  "common_name": 1391,
  "mRNA": 6065,
  "tRNA": 196,
  "ncRNA": 0,
  "rRNA": 0,
  "repeat_region": 0,
  "misc_feature": 0,
  "avg_gene_length": 1552.19,
  "transcript-level": {
    "CDS_transcripts": 6065,
    "CDS_five_utr": 0,
    "CDS_three_utr": 0,
    "CDS_no_utr": 6065,
    "CDS_five_three_utr": 0,
    "CDS_complete": 6039,
    "CDS_no-start": 10,
    "CDS_no-stop": 13,
    "CDS_no-start_no-stop": 3,
    "total_exons": 7079,
    "total_cds_exons": 7079,
    "average_number_transcripts_per_gene": 1.0,
    "multiple_exon_transcript": 786,
    "single_exon_transcript": 5279,
    "average_number_cds_exons": 1.17,
    "avg_exon_length": 1331.36,
    "median_number_exons": 1,
    "max_number_exons": 14,
    "avg_protein_length": 517.37,
    "avg_transcript_length": 1553.95,
    "functional": {
      "go_terms": 0,
      "interproscan": 0,
      "eggnog": 0,
      "pfam": 5119,
      "cazyme": 170,
      "merops": 208,
      "busco": 3254,
      "secretion": 0
    }
  }
}